Erich Schubert: Machine Learning Lecture Recordings

I have uploaded most of my Machine Learning lecture to YouTube. The slides are in English, but the audio is in German. Some very basic contents (e.g., a demo of standard k-means clustering) were left out from this advanced class, and instead only a link to recordings from an earlier class were given. In this class, I wanted to focus on the improved (accelerated) algorithms instead. These are not included here (yet). I believe there are some contents covered in this class you will find nowhere else (yet). The first unit is pretty long (I did not split it further yet). The later units are shorter recordings. ML F1: Principles in Machine Learning

ML F2/F3: Correlation does not Imply Causation & Multiple Testing Problem

ML F4: Overfitting beranpassung

ML F5: Fluch der Dimensionalit t Curse of Dimensionality

ML F6: Intrinsische Dimensionalit t Intrinsic Dimensionality

ML F7: Distanzfunktionen und hnlichkeitsfunktionen

ML L1: Einf hrung in die Klassifikation

ML L2: Evaluation und Wahl von Klassifikatoren

ML L3: Bayes-Klassifikatoren

ML L4: N chste-Nachbarn Klassifikation

ML L5: N chste Nachbarn und Kerndichtesch tzung

ML L6: Lernen von Entscheidungsb umen

ML L7: Splitkriterien bei Entscheidungsb umen

ML L8: Ensembles und Meta-Learning: Random Forests und Gradient Boosting

ML L9: Support Vector Machinen - Motivation

ML L10: Affine Hyperebenen und Skalarprodukte Geometrie f r SVMs

ML L11: Maximum Margin Hyperplane die breitest m gliche Stra e

ML L12: Training Support Vector Machines

ML L13: Non-linear SVM and the Kernel Trick

ML L14: SVM Extensions and Conclusions

ML L15: Motivation of Neural Networks

ML L16: Threshold Logic Units

ML L17: General Artificial Neural Networks

ML L18: Learning Neural Networks with Backpropagation

ML L19: Deep Neural Networks

ML L20: Convolutional Neural Networks

ML L21: Recurrent Neural Networks and LSTM

ML L22: Conclusion Classification

ML U1: Einleitung Clusteranalyse

ML U2: Hierarchisches Clustering

ML U3: Accelerating HAC mit Anderberg s Algorithmus

ML U4: k-Means Clustering

ML U5: Accelerating k-Means Clustering

ML U6: Limitations of k-Means Clustering

ML U7: Extensions of k-Means Clustering

ML U8: Partitioning Around Medoids (k-Medoids)

ML U9: Gaussian Mixture Modeling (EM Clustering)

ML U10: Gaussian Mixture Modeling Demo

Gaussian Mixture Modeling Demo

ML U11: BIRCH and BETULA Clustering

ML U12: Motivation Density-Based Clustering (DBSCAN)

ML U13: Density-reachable and density-connected (DBSCAN Clustering)

ML U14: DBSCAN Clustering

ML U15: Parameterization of DBSCAN

ML U16: Extensions and Variations of DBSCAN Clustering

ML U17: OPTICS Clustering

ML U18: Cluster Extraction from OPTICS Plots

ML U19: Understanding the OPTICS Cluster Order

ML U20: Spectral Clustering

ML U21: Biclustering and Subspace Clustering

ML U22: Further Clustering Approaches

Vishal Gupta: Sikkim 101 for Backpackers

Host to Kanchenjunga, the world s third-highest mountain peak and the endangered Red Panda, Sikkim is a state in northeastern India. Nestled between Nepal, Tibet (China), Bhutan and West Bengal (India), the state offers a smorgasbord of cultures and cuisines. That said, it s hardly surprising that the old spice route meanders through western Sikkim, connecting Lhasa with the ports of Bengal. Although the latter could also be attributed to cardamom (kali elaichi), a perennial herb native to Sikkim, which the state is the second-largest producer of, globally. Lastly, having been to and lived in India, all my life, I can confidently say Sikkim is one of the cleanest & safest regions in India, making it ideal for first-time backpackers.

Brief History

17th century: The Kingdom of Sikkim is founded by the Namgyal dynasty and ruled by Buddhist priest-kings known as the Chogyal.

1890: Sikkim becomes a princely state of British India.

1947: Sikkim continues its protectorate status with the Union of India, post-Indian-independence.

1973: Anti-royalist riots take place in front of the Chogyal's palace, by Nepalis seeking greater representation.

1975: Referendum leads to the deposition of the monarchy and Sikkim joins India as its 22nd state.

Languages

Official: English, Nepali, Sikkimese/Bhotia and Lepcha

Though Hindi and Nepali share the same script (Devanagari), they are not mutually intelligible. Yet, most people in Sikkim can understand and speak Hindi.

Ethnicity

Nepalis: Migrated in large numbers (from Nepal) and soon became the dominant community

Bhutias: People of Tibetan origin. Major inhabitants in Northern Sikkim.

Lepchas: Original inhabitants of Sikkim

Food

Tibetan/Nepali dishes (mostly consumed during winter)
- Thukpa: Noodle soup, rich in spices and vegetables. Usually contains some form of meat. Common variations: Thenthuk and Gyathuk
- Momos: Steamed or fried dumplings, usually with a meat filling.
- Saadheko: Spicy marinated chicken salad.
- Gundruk Soup: A soup made from Gundruk, a fermented leafy green vegetable.
- Sinki : A fermented radish tap-root product, traditionally consumed as a base for soup and as a pickle. Eerily similar to Kimchi.
While pork and beef are pretty common, finding vegetarian dishes is equally easy.
Staple: Dal-Bhat with Subzi. Rice is a lot more common than wheat (rice) possibly due to greater carb content and proximity to West Bengal, India s largest producer of Rice.
Good places to eat in Gangtok
- Hamro Bhansa Ghar, Nimtho (Nepali)
- Taste of Tibet
- Dragon Wok (Chinese & Japanese)

Buddhism in Sikkim

Bayul Demojong (Sikkim), is the most sacred Land in the Himalayas as per the belief of the Northern Buddhists and various religious texts.

Sikkim was blessed by Guru Padmasambhava, the great Buddhist saint who visited Sikkim in the 8th century and consecrated the land.

However, Buddhism is said to have reached Sikkim only in the 17th century with the arrival of three Tibetan monks viz. Rigdzin Goedki Demthruchen, Mon Kathok Sonam Gyaltshen & Rigdzin Legden Je at Yuksom. Together, they established a Buddhist monastery.

In 1642 they crowned Phuntsog Namgyal as the first monarch of Sikkim and gave him the title of Chogyal, or Dharma Raja.

The faith became popular through its royal patronage and soon many villages had their own monastery.

Today Sikkim has over 200 monasteries.

Major monasteries

Rumtek Monastery, 20Km from Gangtok

Lingdum/Ranka Monastery, 17Km from Gangtok

Phodong Monastery, 28Km from Gangtok

Ralang Monastery, 10Km from Ravangla

Tsuklakhang Monastery, Royal Palace, Gangtok

Enchey Monastery, Gangtok

Tashiding Monastery, 35Km from Ravangla

NJP, Gangtok and Darjeeling. They cost about the same as shared cabs but pack in half as many people in luxury cars (Innova, Xylo, etc.) and are hence more comfortable.

Gangtok

Time needed: 1D/1N

Places to visit:

Hanuman Tok

Ganesh Tok

Tashi View Point [6,800ft]

MG Marg

Sikkim Zoo

Gangtok Ropeway

Enchey Monastery

Tsuklakhang Palace & Monastery

Hostels: Tagalong Backpackers (would strongly recommend), Zostel Gangtok

Places to chill: Travel Cafe, Caf Live & Loud and Gangtok Groove

Places to shop: Lal Market and MG Marg

Getting Around

Taxis operate on a reserved or shared basis. In case of the latter, you can pool with other commuters your taxis will pick up and drop en-route.

Naturally shared taxis only operate on popular routes. The easiest way to get around Gangtok is to catch a shared cab from MG Marg.

Reserved taxis for Gangtok sightseeing cost around INR 1000-1500, depending upon the spots you d like to see

Key taxi/bus stands :

Deorali stand: For Darjeeling, Siliguri, Kalimpong

Vajra stand: For North & East Sikkim (Tsomgo Lake & Nathula)

Rumtek taxi: For Ravangla, Pelling, Namchi, Geyzing, Jorethang and Singtam.

Exploring Gangtok on an MTB

If you'd like to explore Gangtok at your own pace, you could rent a cycle from Hub Outdoor or HillBike. I paid INR 600 for a day at HillBike.

These are well-maintained geared mountain bikes, so adapting to inclines and declines should be a breeze, even for beginners.

Cycling down from Ganesh Tok to Tashi View Point is an absolute delight and in my opinion, the best thing you can do in Gangtok.

While it is a little unconventional and exhausting (uphill stretches), but a 100% recommended unless you ve been skipping leg day

Pro tip : Make sure you use the right/back brake while decelerating downhill and not the left/front brake.

Recommended route [3-5hrs, 20.3Km, 1770-2081ft] : Chandmari (HillBike) -> Sera Jhe Dro-Phen-Ling Monastery > Ganesh Tok > Tashi View Point > Bakthung Falls > Vajra Stand > Tsuklakhang Palace & Monastery > Enchey Monastery > Chandmari

North Sikkim

The easiest & most economical way to explore North Sikkim is the 3D/2N package offered by shared-cab drivers.

This includes food, permits, cab rides and accommodation (1N in Lachen and 1N in Lachung)

The accommodation on both nights are at homestays with bare necessities, so keep your hopes low.

In the spirit of sustainable tourism, you ll be asked to discard single-use plastic bottles, so please carry a bottle that you can refill along the way.

Zero Point and Gurdongmer Lake are snow-capped throughout the year

3D/2N Shared-cab Package Itinerary

Day 1

Gangtok (10am) - Chungthang - Lachung (stay)

Day 2

Pre-lunch : Lachung (6am) - Yumthang Valley [12,139ft] - Zero Point - Lachung [15,300ft]

Post-lunch : Lachung - Chungthang - Lachen (stay)

Day 3

Pre-lunch : Lachen (5am) - Kala Patthar - Gurdongmer Lake [16,910ft] - Lachen

Post-lunch : Lachen - Chungthang - Gangtok (7pm)

This itinerary is idealistic and depends on the level of snowfall.

Some drivers might switch up Day 2 and 3 itineraries by visiting Lachen and then Lachung, depending upon the weather.

Areas beyond Lachen & Lachung are heavily militarized since the Indo-China border is only a few miles away.

East Sikkim

Zuluk and Silk Route

Time needed: 2D/1N

Zuluk [9,400ft] is a small hamlet with an excellent view of the eastern Himalayan range including the Kanchenjunga.

Was once a transit point to the historic Silk Route from Tibet (Lhasa) to India (West Bengal).

The drive from Gangtok to Zuluk takes at least four hours. Hence, it makes sense to spend the night at a homestay and space out your trip to Zuluk

Tsomgo Lake and Nathula

Time Needed : 1D

A Protected Area Permit is required to visit these places, due to their proximity to the Chinese border

Tsomgo/Chhangu Lake [12,313ft]

Glacial lake, 40 km from Gangtok.

Remains frozen during the winter season.

You can also ride on the back of a Yak for INR 300

Baba Mandir

An old temple dedicated to Baba Harbhajan Singh, a Sepoy in the 23rd Regiment, who died in 1962 near the Nathu La during Indo China war.

Nathula Pass [14,450ft]

Located on the Indo-Tibetan border crossing of the Old Silk Route, it is one of the three open trading posts between India and China.

Plays a key role in the Sino-Indian Trade and also serves as an official Border Personnel Meeting(BPM) Point.

May get cordoned off by the Indian Army in event of heavy snowfall or for other security reasons.

West Sikkim

Time needed: 3N/1N

Hostels at Pelling : Mochilerro Ostillo

Itinerary

Day 1: Gangtok - Ravangla - Pelling

Leave Gangtok early, for Ravangla through the Temi Tea Estate route.

Spend some time at the tea garden and then visit Buddha Park at Ravangla

Head to Pelling from Ravangla

Day 2: Pelling sightseeing

Hire a cab and visit Skywalk, Pemayangtse Monastery, Rabdentse Ruins, Kecheopalri Lake, Kanchenjunga Falls.

Day 3: Pelling - Gangtok/Siliguri

Wake up early to catch a glimpse of Kanchenjunga at the Pelling Helipad around sunrise

Head back to Gangtok on a shared-cab

You could take a bus/taxi back to Siliguri if Pelling is your last stop.

Darjeeling

In my opinion, Darjeeling is lovely for a two-day detour on your way back to Bagdogra/Siliguri and not any longer (unless you re a Bengali couple on a honeymoon)

Once a part of Sikkim, Darjeeling was ceded to the East India Company after a series of wars, with Sikkim briefly receiving a grant from EIC for gifting Darjeeling to the latter

Post-independence, Darjeeling was merged with the state of West Bengal.

Itinerary

Day 1 :

Take a cab from Gangtok to Darjeeling (shared-cabs cost INR 300 per seat)

Reach Darjeeling by noon and check in to your Hostel. I stayed at Hideout.

Spend the evening visiting either a monastery (or the Batasia Loop), Nehru Road and Mall Road.

Grab dinner at Glenary whilst listening to live music.

Day 2:

Wake up early to catch the sunrise and a glimpse of Kanchenjunga at Tiger Hill. Since Tiger Hill is 10km from Darjeeling and requires a permit, book your taxi in advance.

Alternatively, if you don t want to get up at 4am or shell out INR1500 on the cab to Tiger Hill, walk to the Kanchenjunga View Point down Mall Road

Next, queue up outside Keventers for breakfast with a view in a century-old cafe

Get a cab at Gandhi Road and visit a tea garden (Happy Valley is the closest) and the Ropeway. I was lucky to meet 6 other backpackers at my hostel and we ended up pooling the cab at INR 200 per person, with INR 1400 being on the expensive side, but you could bargain.

Get lunch, buy some tea at Golden Tips, pack your bags and hop on a shared-cab back to Siliguri. It took us about 4hrs to reach Siliguri, with an hour to spare before my train.

If you ve still got time on your hands, then check out the Peace Pagoda and the Darjeeling Himalayan Railway (Toy Train). At INR 1500, I found the latter to be too expensive and skipped it.

Tips and hacks

Download offline maps, especially when you re exploring Northern Sikkim.

Food and booze are the cheapest in Gangtok. Stash up before heading to other regions.

Keep your Aadhar/Passport handy since you need permits to travel to North & East Sikkim.

In rural areas and some cafes, you may get to try Rhododendron Wine, made from Rhododendron arboreum a.k.a Gurans. Its production is a little hush-hush since the flower is considered holy and is also the National Flower of Nepal.

If you don t want to invest in a new jacket, boots or a pair of gloves, you can always rent them at nominal rates from your hotel or little stores around tourist sites.

Check the weather of a region before heading there. Low visibility and precipitation can quite literally dampen your experience.

Keep your itinerary flexible to accommodate for rest and impromptu plans.

Shops and restaurants close by 8pm in Sikkim and Darjeeling. Plan for the same.

Carry

a couple of extra pairs of socks (woollen, if possible)

a pair of slippers to wear indoors

a reusable water bottle

an umbrella

a power bank

a couple of tablets of Diamox. Helps deal with altitude sickness

extra clothes and wet bags since you may not get a chance to wash/dry your clothes

a few passport size photographs

Shared-cab hacks

Intercity rides can be exhausting. If you can afford it, pay for an additional seat.

Call shotgun on the drives beyond Lachen and Lachung. The views are breathtaking.

Return cabs tend to be cheaper (WB cabs travelling from SK and vice-versa)

Cost

My median daily expenditure (back when I went to Sikkim in early March 2021) was INR 1350.

This includes stay (bunk bed), food, wine and transit (shared cabs)

In my defence, I splurged on food, wine and extra seats in shared cabs, but if you re on a budget, you could easily get by on INR 1 - 1.2k per day.

For a 9-day trip, I ended up shelling out nearly INR 15k, including 2AC trains to & from Kolkata

Note : Summer (March to May) and Autumn (October to December) are peak seasons, and thereby more expensive to travel around.

Souvenirs and things you should buy

Buddhist souvenirs :

Colourful Prayer Flags (great for tying on bikes or behind car windshields)

Miniature Prayer/Mani Wheels

Lucky Charms, Pendants and Key Chains

Cham Dance masks and robes

Singing Bowls

Common symbols: Om mani padme hum, Ashtamangala, Zodiac signs

Handicrafts & Handlooms

Tibetan Yak Wool shawls, scarfs and carpets

Sikkimese Ceramic cups

Thangka Paintings

Edibles

Darjeeling Tea (usually brewed and not boiled)

Wine (Arucha Peach & Rhododendron)

Dalle Khursani (Chilli) Paste and Pickle

Header Icon made by Freepik from www.flaticon.com is licensed by CC 3.0 BY

Timo Jyrinki: MotionPhoto / MicroVideo File Formats on Pixel Phones

Google Pixel phones support what they call Motion Photo which is essentially a photo with a short video clip attached to it. They are quite nice since they bring the moment alive, especially as the capturing of the video starts a small moment before the shutter button is pressed. For most viewing programs they simply show as static JPEG photos, but there is more to the files.

I d really love proper Shotwell support for these file formats, so I posted a longish explanation with many of the details in this blog post to a ticket there too. Examples of the newer format are linked there too.

There are actually two different formats, an old one that is already obsolete, and a newer current format. The older ones are those that your Pixel phone recorded as MVIMG_[datetime].jpg", and they have the following meta-data:

Xmp.GCamera.MicroVideo                       XmpText     1  1
Xmp.GCamera.MicroVideoVersion                XmpText     1  1
Xmp.GCamera.MicroVideoOffset                 XmpText     7  4022143
Xmp.GCamera.MicroVideoPresentationTimestampUs XmpText     7  1331607

The offset is actually from the end of the file, so one needs to calculate accordingly. But it is exact otherwise, so one simply extract a file with that meta-data information:

#!/bin/bash
#
# Extracts the microvideo from a MVIMG_*.jpg file

# The offset is from the ending of the file, so calculate accordingly
offset=$(exiv2 -p X "$1"   grep MicroVideoOffset   sed 's/.*\"\(.*\)"/\1/')
filesize=$(du --apparent-size --block=1 "$1"   sed 's/^\([0-9]*\).*/\1/')
extractposition=$(expr $filesize - $offset)
echo offset: $offset
echo filesize: $filesize
echo extractposition=$extractposition
dd if="$1" skip=1 bs=$extractposition of="$(basename -s .jpg $1).mp4"

The newer format is recorded in filenames called PXL_[datetime].MP.jpg , and they have a _lot_ of additional metadata:

Xmp.GCamera.MotionPhoto                      XmpText     1  1
Xmp.GCamera.MotionPhotoVersion               XmpText     1  1
Xmp.GCamera.MotionPhotoPresentationTimestampUs XmpText     6  233320
Xmp.xmpNote.HasExtendedXMP                   XmpText    32  E1F7505D2DD64EA6948D2047449F0FFA
Xmp.Container.Directory                      XmpText     0  type="Seq"
Xmp.Container.Directory[1]                   XmpText     0  type="Struct"
Xmp.Container.Directory[1]/Container:Item    XmpText     0  type="Struct"
Xmp.Container.Directory[1]/Container:Item/Item:Mime XmpText    10  image/jpeg
Xmp.Container.Directory[1]/Container:Item/Item:Semantic XmpText     7  Primary
Xmp.Container.Directory[1]/Container:Item/Item:Length XmpText     1  0
Xmp.Container.Directory[1]/Container:Item/Item:Padding XmpText     1  0
Xmp.Container.Directory[2]                   XmpText     0  type="Struct"
Xmp.Container.Directory[2]/Container:Item    XmpText     0  type="Struct"
Xmp.Container.Directory[2]/Container:Item/Item:Mime XmpText     9  video/mp4
Xmp.Container.Directory[2]/Container:Item/Item:Semantic XmpText    11  MotionPhoto
Xmp.Container.Directory[2]/Container:Item/Item:Length XmpText     7  1679555
Xmp.Container.Directory[2]/Container:Item/Item:Padding XmpText     1  0

Sounds like fun and lots of information. However I didn t see why the length in first item is 0 and I didn t see how to use the latter Length info. But I can use the mp4 headers to extract it:

#!/bin/bash
#
# Extracts the motion part of a MotionPhoto file PXL_*.MP.mp4

extractposition=$(grep --binary --byte-offset --only-matching --text \
-P "\x00\x00\x00\x18\x66\x74\x79\x70\x6d\x70\x34\x32" $1   sed 's/^\([0-9]*\).*/\1/')

dd if="$1" skip=1 bs=$extractposition of="$(basename -s .jpg $1).mp4"

UPDATE: I wrote most of this blog post earlier. When now actually getting to publishing it a week later, I see the obvious ie the Length is again simply the offset from the end of the file so one could do the same less brute force approach as for MVIMG. I ll leave the above as is however for the of binary grepping.(cross-posted to my other blog)

Louis-Philippe V ronneau: New Year, New OpenPGP Key

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Sun, 07 Mar 2021 13:00:17 -0500
I've recently set up a new OpenPGP key and will be transitioning away from my
old one.
It is a chance for me to start using a OpenPGP hardware token and to transition
to a new personal email address (my main public contact is still my
 @debian.org  address).
Please note that I've partially redacted some email addresses from this
statement to minimise the amount of spam I receive. It shouldn't be hard for
actual humans to follow the instructions below to find the complete addresses.
The old key will continue to be valid for a few months, but will eventually be
revoked.
You might know my old OpenPGP certificate as:
pub   rsa4096/0x7AEAC4EC6AAA0A97 2014-12-22 [expires: 2021-06-02]
      Key fingerprint = 677F 54F1 FA86 81AD 8EC0  BCE6 7AEA C4EC 6AAA 0A97
uid       Louis-Philippe V ronneau <REDACTED@riseup.net>
uid       Louis-Philippe V ronneau (alias) <REDACTED@riseup.net>
uid       Louis-Philippe V ronneau (debian) <REDACTED@debian.org>
My new OpenPGP certificate is:
pub   ed25519/0xE1E5457C8BAD4113 2021-03-06 [expires: 2022-03-06]
      Key fingerprint = F64D 61D3 21F3 CB48 9156  753D E1E5 457C 8BAD 4113
uid       Louis-Philippe V ronneau <REDACTED@veronneau.org>
uid       Louis-Philippe V ronneau <REDACTED@debian.org>
These days, I mostly use my key for Debian and to sign git commit. I don't
really expect you to sign my new key if you had signed my old one.
I've published the new certificate on keys.openpgp.org as well as on my
personal website. You can fetch it like this:
    $ wget -O- https://veronneau.org/media/openpgp.key   gpg --import
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEEZ39U8fqGga2OwLzmeurE7GqqCpcFAmBFFM8ACgkQeurE7Gqq
CpcuchAAscAeszdtA+TlCI4YvK5nlk+nJnCnNBSnl7Et+jiNjq8kB/Fud+dWMTXC
Zag8oJkalbbxub0BT0bEAn+BiBunu58E0gd0Xq4syTbqZ5o5IN17S/tfxCD0k1hf
ewrnYZ2l0i5g4YvHGKC+Xv4D+Z84BylnIRaXHqlUdluOVfVYDfLybOAqoktO/KUH
I+vQBwXj0Fr/QAtgiz5Nwh/YHFiU9xMSvr5ozRwAFs6+xfIqFHuVPRRkEN5iVo4D
kkMIz+kFfkoh4aWIP4dgAu39XnEgxwTR9J+4yE8TzCCMzO7xCK0X6vqgPAxYMPvb
RuP4FnGWOnGnlcudCUAUkOaryrwRi+dPQTnNICHTYsvVc7dg+W0EhVUkwEuuEwpI
qtcB/Y5AGhqK0Cc11uXiFjIQwLTgwcUez4F0xrGeqsTtAM5gyRup2w0jbocTuYSh
ZRv/2zwrq/S3xVrUYGqdT+L5odmkBzz9zOwY5WlU2H9CMFOdh71XOv9wWQXan9ou
hLRodeOQ8MinIBP+sX36ol1zg/aP7mCHvRRSBzWt7l3WhVxgZFpNwIfp/RZqU0R4
IEq48mntFhPvHJjFmAKLKK/ckzNMtSn+HWQPJV3HTInKCTu5PTNMU3SAvPHOHEps
V6WWSOPB+1Lm/tlIULDc+0SopWoiWO4NObCSs8zMZHlYPBk5x/KIdQQBFgoAHRYh
BMqnQAcHqBawIC/DzfQlelCyHPqFBQJgRRTPAAoJEPQlelCyHPqFFVEA/1qScaAk
O+eBEE4q0BaJDsqweCS1XCcuQGkQCKi5Zv6kAQChQ96Ve7cKbN/wRkT9pdIhmx01
+CmIsnp3k6N0ZYLLCg==
=onl0
-----END PGP SIGNATURE-----

Adnan Hodzic: Degiro trading tracker Simplified tracking of your investments

TL;DRVisit degiro-trading-tracker on Github I was always interested in stocks and investing. While I wanted to get into trading for long time, I could never... The post Degiro trading tracker Simplified tracking of your investments appeared first on FoolControl: Phear the penguin.

Vincent Bernat: Running Isso on NixOS in a Docker container

This short article documents how I run Isso, the commenting system used by this blog, inside a Docker container on NixOS, a Linux distribution built on top of Nix. Nix is a declarative package manager for Linux and other Unix systems.

While NixOS 20.09 includes a derivation for Isso, it is unfortunately broken and relies on Python 2. As I am also using a fork of Isso, I have built my own derivation, heavily inspired by the one in master:¹

issoPackage = with pkgs.python3Packages; buildPythonPackage rec  
  pname = "isso";
  version = "custom";
  src = pkgs.fetchFromGitHub  
    # Use my fork
    owner = "vincentbernat";
    repo = pname;
    rev = "vbe/master";
    sha256 = "0vkkvjcvcjcdzdj73qig32hqgjly8n3ln2djzmhshc04i6g9z07j";
   ;
  propagatedBuildInputs = [
    itsdangerous
    jinja2
    misaka
    html5lib
    werkzeug
    bleach
    flask-caching
  ];
  buildInputs = [
    cffi
  ];
  checkInputs = [ nose ];
  checkPhase = ''
    $ python.interpreter  setup.py nosetests
  '';
 ;

I want to run Isso through Gunicorn. To this effect, I build a Python environment combining Isso and Gunicorn. Then, I can invoke the latter with "$ issoEnv /bin/gunicorn", like with a virtual environment.

issoEnv = pkgs.python3.buildEnv.override  
    extraLibs = [
      issoPackage
      pkgs.python3Packages.gunicorn
      pkgs.python3Packages.gevent
    ];
 ;

Before building a Docker image, I also need to specify the Isso configuration file for Isso:

issoConfig = pkgs.writeText "isso.conf" ''
  [general]
  dbpath = /db/comments.db
  host =
    https://vincent.bernat.ch
    http://localhost:8080
  notify = smtp
  [ ]
'';

NixOS comes with a convenient tool to build a Docker image without a Dockerfile:

issoDockerImage = pkgs.dockerTools.buildImage  
  name = "isso";
  tag = "latest";
  extraCommands = ''
    mkdir -p db
  '';
  config =  
    Cmd = [ "$ issoEnv /bin/gunicorn"
            "--name" "isso"
            "--bind" "0.0.0.0:$ port "
            "--worker-class" "gevent"
            "--workers" "2"
            "--worker-tmp-dir" "/dev/shm"
            "--preload"
            "isso.run"
          ];
    Env = [
      "ISSO_SETTINGS=$ issoConfig "
      "SSL_CERT_FILE=$ pkgs.cacert /etc/ssl/certs/ca-bundle.crt"
    ];
   ;
 ;

Because we refer to the issoEnv derivation in config.Cmd, the whole derivation, including Isso and Gunicorn, is copied inside the Docker image. The same applies for issoConfig, the configuration file we created earlier, and pkgs.cacert, the derivation containing trusted root certificates. The resulting image is 171 MB once installed, which is comparable to the Debian Buster image generated by the official Dockerfile. NixOS features an abstraction to run Docker containers. It is not currently documented in NixOS manual but you can look at the source code of the module for the available options. I choose to use Podman instead of Docker as the backend because it does not require running an additional daemon.

virtualisation.oci-containers =  
  backend = "podman";
  containers =  
    isso =  
      image = "isso";
      imageFile = issoDockerImage;
      ports = ["127.0.0.1:$ port :$ port "];
      volumes = [
        "/var/db/isso:/db"
      ];
     ;
   ;
 ;

A systemd unit file is automatically created to run and supervise the container:

$ systemctl status podman-isso.service
  podman-isso.service
     Loaded: loaded (/nix/store/a66gzqqwcdzbh99sz8zz5l5xl8r8ag7w-unit->
     Active: active (running) since Sun 2020-11-01 16:04:16 UTC; 4min 44s ago
    Process: 14564 ExecStartPre=/nix/store/95zfn4vg4867gzxz1gw7nxayqcl>
   Main PID: 14697 (podman)
         IP: 0B in, 0B out
      Tasks: 10 (limit: 2313)
     Memory: 221.3M
        CPU: 10.058s
     CGroup: /system.slice/podman-isso.service
              14697 /nix/store/pn52xgn1wb2vr2kirq3xj8ij0rys35mf-podma>
              14802 /nix/store/7vsba54k6ag4cfsfp95rvjzqf6rab865-conmo>
nov. 01 16:04:17 web03 podman[14697]: container init (image=localhost/isso:latest)
nov. 01 16:04:17 web03 podman[14697]: container start (image=localhost/isso:latest)
nov. 01 16:04:17 web03 podman[14697]: container attach (image=localhost/isso:latest)
nov. 01 16:04:19 web03 conmon[14802]: INFO: connected to SMTP server
nov. 01 16:04:19 web03 conmon[14802]: INFO: connected to https://vincent.bernat.ch
nov. 01 16:04:19 web03 conmon[14802]: [INFO] Starting gunicorn 20.0.4
nov. 01 16:04:19 web03 conmon[14802]: [INFO] Listening at: http://0.0.0.0:8080 (1)
nov. 01 16:04:19 web03 conmon[14802]: [INFO] Using worker: gevent
nov. 01 16:04:19 web03 conmon[14802]: [INFO] Booting worker with pid: 8
nov. 01 16:04:19 web03 conmon[14802]: [INFO] Booting worker with pid: 9

As the last step, we configure Nginx to forward requests for comments.luffy.cx to the container. NixOS provides a simple integration to grab a Let s Encrypt certificate.

services.nginx.virtualHosts."comments.luffy.cx" =  
  root = "/data/webserver/comments.luffy.cx";
  enableACME = true;
  forceSSL = true;
  extraConfig = ''
    access_log /var/log/nginx/comments.luffy.cx.log anonymous;
  '';
  locations."/" =  
    proxyPass = "http://127.0.0.1:$ port ";
    extraConfig = ''
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $host;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_hide_header Set-Cookie;
      proxy_hide_header X-Set-Cookie;
      proxy_ignore_headers Set-Cookie;
    '';
   ;
 ;
security.acme.certs."comments.luffy.cx" =  
  email = lib.concatStringsSep "@" [ "letsencrypt" "vincent.bernat.ch" ];
 ;

While I still struggle with Nix and NixOS, I am convinced this is how declarative infrastructure should be done. I like how in one single file, I can define the derivation to build Isso, the configuration, the Docker image, the container definition, and the Nginx configuration. The Nix language is used both for building packages and for managing configurations. Moreover, the Docker image is updated automatically like a regular NixOS host. This solves an issue plaguing the Docker ecosystem: no more stale images! My next step would be to combine this approach with Nomad, a simple orchestrator to deploy and manage containers.

There is a subtle difference: I am using buildPythonPackage instead of buildPythonApplication. This is important for the next step. I didn t investigate if an application can be converted to a package easily.

Antoine Beaupr : Presentation tools

I keep forgetting how to make presentations. I had a list of tools in a wiki from a previous job, but that's now private and I don't see why I shouldn't share this (even if for myself!). So here it is. What's your favorite presentation tool?

Tips

if you have some text to present, outline keywords so that you can present your subject without reading every word

ideally, don't read from your slides - they are there to help people follow, not for people to read

even better: make your slides pretty with only a few words, or don't make slides at all

Further advice:

7 tips by Jeffrey Veen

10 tips by Neil Patel

The Art of Presenting by Matt Westgate (video)

Presenting You by Emma Jane Hogbin (video)

I'm currently using Pandoc with PDF input (with a trip through LaTeX) for most slides, because PDFs are more reliable and portable than web pages. I've also used Libreoffice, Pinpoint, and S5 (through RST) in the past. I miss Pinpoint, too bad that it died. Some of my presentations are available in my GitLab.com account:

Presentations while I worked at Koumbit

Short presentation about PRISM

Security training

Ethics in computing, based on this blog post

Presentation about the Maple Spring, at OHM2013

First presentation at Tor

See also my list of talks and presentations which I can't seem to keep up to date.

Tools

Beamer (LaTeX)

LaTeX class

Do not use directly unless you are a LaTeX expert or masochist, see Pandoc below

see also powerdot

Home page

Darkslide

HTML, Javascript

presenter notes, table of contents, Markdown, RST, Textile, themes, code samples, auto-reload

Home page, demo

Impress.js

Javascript

Zooms in and out, 3D support

Source code, demo

Hekyll uses Jekyll as a backend

Impressive

simply displays PDFs or images

page transitions, overview screen, highlighting

Home page

Libreoffice Impress

Powerpoint clone

Makes my life miserable

PDF export, presenter notes, outline view, etc

Home page, screenshots

Magicpoint

ancestor of everyone else (1997!)

text input format, image support, talk timer, slide guides, HTML/Postscript export, draw on slides, X11 output

no release since 2008

Home page

mdp and lookatme (commandline)

Commandline-only, markdown

Home page

lookatme is similar

Pandoc

Allows converting from basically whatever into slides, including Beamer, DZSlides, reveal.js, slideous, slidy, Powerpoint

PDF, HTML, Powerpoint export, presentation notes, full screen background images

nice plain text or markdown input format

Home page, documentation

PDF Presenter

PDF presentation tool, shows presentation notes

basically "Keynote for Linux"

Home page, pdf-presenter-console in Debian

Pinpoint

Native GNOME app

Full screen slides, PDF export, live change, presenter notes, pango markup, video, image backgrounds

Home page

Abandoned since at least 2019

Reveal.js

HTML, Javascript

PDF export, Markdown, LaTeX support, syntax-highlighting, nested slides, speaker notes

Source code, demo

S5

HTML, CSS

incremental, bookmarks, keyboard controls

can be transformed from ReStructuredText (RST) with `rst2s5` with python-docutils

Home page, demo

sent

X11 only

plain text, black on white, image support, and that's it

from the suckless.org elitists

Home page

Sozi

Entire presentation is one poster, zooming and jumping around

SVG + Javascript

Home page, demo

Other options Another option I have seriously considered is just generate a series of images with good resolution, hopefully matching the resolution (or at least aspect ratio) of the output device. Then you flip through a series of images one by one. In that case, any of those image viewers (not an exhaustive list) would work:

Geeqie

GNOME's eog

pho

feh

fim

sxiv

Update: it turns out I already wrote a somewhat similar thing when I did a recent presentation. If you're into rants, you might enjoy the README file accompanying the Kubecon rant presentation. TL;DR: "makes me want to scream" and "yet another unsolved problem space, sigh" (refering to "display images full-screen" specifically).

Lisandro Dami n Nicanor P rez Meyer: Simplified Monitoring of Patients in Situations of Mass Hospitalization (MoSimPa) - Fighting COVID-19

I have been quite absent from Debian stuff lately, but this increased since COVID-19 hits us. In this blog post I'll try to sketch what I have been doing to help fight COVID-19 this last few months.

In the beginningWhen the pandemic reached Argentina the government started a quarantine. We engineers (like engineers around the world) started to think on how to put our abilities in order to help with the situation. Some worked toward providing more protection elements to medical staff, some towards increasing the number of ventilation machines at disposal. Another group of people started thinking on another ways of helping. In Bah a Blanca arised the idea of monitoring some variables remotely and in masse.

Simplified Monitoring of Patients in Situations of Mass Hospitalization (MoSimPa)

This is where the idea of remotely monitored devices came in, and MoSimPa (from the spanish of "monitoreo simplificado de pacientes en situaci n de internaci n masiva") started to get form. The idea is simple: oximetry (SpO2), heart rate and body temperature will be recorded and, instead of being shown in a display in the device itself, they will be transmitted and monitored in one or more places. In this way medical staff doesn't has to reach a patient constantly and monitoring could be done by medical staff for more patients at the same time. In place monitoring can also happen using a cellphone or tablet.

The devices do not have a screen of their own and almost no buttons, making them more cheap to build and thus more in line with the current economic reality of Argentina.

This is where the project Para Ayudar was created. The project aims to produce the aforementioned non-invasive device to be used in health institutions, hospitals, intra hospital transports and homes.

It is worth to note that the system is designed as a complementary measure for continuous monitoring of a pacient. Care should be taken to check that symptomps and overall patient status don't mean an inmediate life threat. In other words, it is NOT designed for ICUs.

All the above done with Free/Libre/Open Source software and hardware designs. Any manufacturing company can then use them for mass production.

The importance of early pneumonia detection

We were already working in MoSimPa when an NYTimes article caught or attention: "The Infection That s Silently Killing Coronavirus Patients". From the article:

A vast majority of Covid pneumonia patients I met had remarkably low oxygen saturations at triage seemingly incompatible with life but they were using their cellphones as we put them on monitors. Although breathing fast, they had relatively minimal apparent distress, despite dangerously low oxygen levels and terrible pneumonia on chest X-rays.

This greatly reinforced the idea we were on the right track.

The project from a technical standpoint

As the project is primarily designed for and by Argentinians the current system design and software documentation is written in spanish, but the source code (or at least most of it) is written in english. Should anyone need it in english please do not hesitate in asking me.
General system description

The system is comprised of the devices, a main machine acting as a server (in our case for small setups a Raspberry Pi) and the possibility of accessing data trough cell phones, tablets or other PCs in the network.
The hardware

As of today this is the only part in which I still can't provide schematics, but I'll update this blog post and technical doc with them as soon as I get my hands into them.

Again the design is due to be built in Argentina where getting our hands on hardware is not easy. Moreover it needs to be as cheap as possible, specially now that the Argentinian currency, the peso, is every day more depreciated. So we decided on using an ESP32 as the main microprocessor and a set of Maxim sensors devices. Again, more info when I have them at hand.

The software

Here we have many more components to describe. Firstly the ESP32 code is done with the Arduino SDK. This part of the stack will receive many updates soon, as soon as the first hardware prototypes are out.

For the rest of the stack I decided to go ahead with whatever is available in Debian stable. Why? Well, Raspbian provides a Debian stable-based image and I'm a Debian Developer, so things should go just natural for me in that front. Of course each component has its own packaging. I'm one of Debian's Qt maintainers then using Qt will also be quite natural for me. Plots? Qwt, of course. And with that I have most of my necessities fulfilled. I choose PostgreSql as database server and Mosquitto as MQTT broker.

Between the database and MQTT is mosimpa-datakeeper. The piece of software from which medical staff monitor patients is unsurprisingly called mosimpa-monitor.

MoSimPa's monitor main screen

Plots of a patient's data

Alarm thresholds setup

And for managing patients, devices, locations and internments (CRUD anyone?) there is currently a Qt-based application called mosimpa-abm.

ABM main screen

ABM internments view

The idea is to replace it with a web service so it doesn't needs to be confined to the RPi or require installations in other machines. I considered using webassembly but I would have to also build PostgreSql in order to compile Qt's plugin.

Translations? Of course! As I have already mentioned the code is written in English. Qt allows to easily translate applications, so I keep a Spanish one as the code changes (and we are primarily targeting spanish-speaking people). But of course this also means it can be easily translated to whichever language is necessary.

Even if I am a packager I still have some stuff to fix from the packaging itself, like letting datakeeper run with its own user. I just haven't got to it yet.

A video showing the software in action

Certifications

We are working towards getting the system certified by ANMAT, which is the Argentinian equivalent for EEUU's FDA.

Funding

While all the people involved are working ad-honorem funding is still required in order to buy materials, create the prototypes, etc. The project created payments links with Mercado Pago (in spanish and argentinian pesos) and other bank methods (PDF, also in spanish).

I repeat the links here with an aproximation to US$.

- 500 AR$ (less than 8 US$)
- 1000 AR$ (less than 15 US$)
- 2000 AR$ (less than 30 US$)
- 3000 AR$ (less than 45 US$)
- 5000 AR$ (less than 75 US$)

You can check the actual convertion rate in https://www.google.com/search?q=argentine+peso+to+us+dollars

The project was also presented at a funding call of argentinian Agencia de Promoci n de la Investigaci n, el Desarrollo Tecnol gico y la Innovaci n (Agencia I+D+i). 900+ projects where presented and 64 funded, MoSimPa between them.

Matthew Palmer: Private Key Redaction: UR DOIN IT RONG

Because posting private keys on the Internet is a bad idea, some people like to redact their private keys, so that it looks kinda-sorta like a private key, but it isn t actually giving away anything secret. Unfortunately, due to the way that private keys are represented, it is easy to redact a key in such a way that it doesn t actually redact anything at all. RSA private keys are particularly bad at this, but the problem can (potentially) apply to other keys as well. I ll show you a bit of Inside Baseball with key formats, and then demonstrate the practical implications. Finally, we ll go through a practical worked example from an actual not-really-redacted key I recently stumbled across in my travels.

The Private Lives of Private Keys Here is what a typical private key looks like, when you come across it:

Obviously, there s some hidden meaning in there computers don t encrypt things by shouting BEGIN RSA PRIVATE KEY! , after all. What is between the BEGIN/END lines above is, in fact, a base64-encoded DER format ASN.1 structure representing a PKCS#1 private key. In simple terms, it s a list of numbers very important numbers. The list of numbers is, in order:

A version number (0);
The public modulus , commonly referred to as n ;
The public exponent , or e (which is almost always 65,537, for various unimportant reasons);
The private exponent , or d ;
The two private primes , or p and q ;
Two exponents, which are known as dmp1 and dmq1 ; and
A coefficient, known as iqmp .

Why Is This a Problem? The thing is, only three of those numbers are actually required in a private key. The rest, whilst useful to allow the RSA encryption and decryption to be more efficient, aren t necessary. The three absolutely required values are e, p, and q. Of the other numbers, most of them are at least about the same size as each of p and q. So of the total data in an RSA key, less than a quarter of the data is required. Let me show you with the above toy key, by breaking it down piece by piece¹:

MGI DER for this is a sequence
CAQ version (0)
CxjdTmecltJEz2PLMpS4BX n
AgMBAA e
ECEDKtuwD17gpagnASq1zQTY d
ECCQDVTYVsjjF7IQ p
IJANUYZsIjRsR3 q
AgkAkahDUXL0RS dmp1
ECCB78r2SnsJC9 dmq1
AghaOK3FsKoELg== iqmp

Remember that in order to reconstruct all of these values, all I need are e, p, and q and e is pretty much always 65,537. So I could redact almost all of this key, and still give all the important, private bits of this key. Let me show you:

-----BEGIN RSA PRIVATE KEY-----
..............................................................EC
CQDVTYVsjjF7IQIJANUYZsIjRsR3....................................
........
-----END RSA PRIVATE KEY-----

Now, I doubt that anyone is going to redact a key precisely like this but then again, this isn t a typical RSA key. They usually look a lot more like this:

People typically redact keys by deleting whole lines, and usually replacing them with [...] and the like. But only about 345 of those 1588 characters (excluding the header and footer) are required to construct the entire key. You can redact about 4/5ths of that giant blob of stuff, and your private parts (or at least, those of your key) are still left uncomfortably exposed.

But Wait! There s More! Remember how I said that everything in the key other than `e`, `p`, and `q` could be derived from those three numbers? Let s talk about one of those numbers: `n`. This is known as the public modulus (because, along with `e`, it is also present in the public key). It is very easy to calculate: `n = p * q`. It is also very early in the key (the second number, in fact). Since `n = p * q`, it follows that `q = n / p`. Thus, as long as the key is intact up to `p`, you can derive `q` by simple division.

Real World Redaction At this point, I d like to introduce an acquaintance of mine: Mr. Johan Finn. He is the proud owner of the GitHub repo johanfinn/scripts. For a while, his repo contained a script that contained a poorly-redacted private key. He since deleted it, by making a new commit, but of course because git never really deletes anything, it s still available. Of course, Mr. Finn may delete the repo, or force-push a new history without that commit, so here is the redacted private key, with a bit of the surrounding shell script, for our illustrative pleasure:

#Add private key to .ssh folder
cd /home/johan/.ssh/
echo  "-----BEGIN RSA PRIVATE KEY-----
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
 
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::.::
:::::::::::::::::::::::::::.::::::::::::::::::::::::::::::::::::
LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLlL
 
 
 
YYYYYYYYYYYYYYYYYYYYYyYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
-----END RSA PRIVATE KEY-----" >> id_rsa

Now, if you try to reconstruct this key by removing the obvious garbage lines (the ones that are all repeated characters, some of which aren t even valid base64 characters), it still isn t a key at least, openssl pkey doesn t want anything to do with it. The key is very much still in there, though, as we shall soon see. Using a gem I wrote and a quick bit of Ruby, we can extract a complete private key. The irb session looks something like this:

>> require "derparse"
>> b64 = <<EOF
MIIJKgIBAAKCAgEAxEVih1JGb8gu/Fm4AZh+ZwJw/pjzzliWrg4mICFt1g7SmIE2
TCQMKABdwd11wOFKCPc/UzRH/fHuQcvWrpbOSdqev/zKff9iedKw/YygkMeIRaXB
fYELqvUAOJ8PPfDm70st9GJRhjGgo5+L3cJB2gfgeiDNHzaFvapRSU0oMGQX+kI9
ezsjDAn+0Pp+r3h/u1QpLSH4moRFGF4omNydI+3iTGB98/EzuNhRBHRNq4oBV5SG
Pq/A1bem2ninnoEaQ+OPESxYzDz3Jy9jV0W/6LvtJ844m+XX69H5fqq5dy55z6DW
sGKn78ULPVZPsYH5Y7C+CM6GAn4nYCpau0t52sqsY5epXdeYx4Dc+Wm0CjXrUDEe
Egl4loPKDxJkQqQ/MQiz6Le/UK9vEmnWn1TRXK3ekzNV4NgDfJANBQobOpwt8WVB
rbsC0ON7n680RQnl7PltK9P1AQW5vHsahkoixk/BhcwhkrkZGyDIl9g8Q/Euyoq3
eivKPLz7/rhDE7C1BzFy7v8AjC3w7i9QeHcWOZFAXo5hiDasIAkljDOsdfD4tP5/
wSO6E6pjL3kJ+RH2FCHd7ciQb+IcuXbku64ln8gab4p8jLa/mcMI+V3eWYnZ82Yu
axsa85hAe4wb60cp/rCJo7ihhDTTvGooqtTisOv2nSvCYpcW9qbL6cGjAXECAwEA
AQKCAgEAjz6wnWDP5Y9ts2FrqUZ5ooamnzpUXlpLhrbu3m5ncl4ZF5LfH+QDN0Kl
KvONmHsUhJynC/vROybSJBU4Fu4bms1DJY3C39h/L7g00qhLG7901pgWMpn3QQtU
4P49qpBii20MGhuTsmQQALtV4kB/vTgYfinoawpo67cdYmk8lqzGzzB/HKxZdNTq
s+zOfxRr7PWMo9LyVRuKLjGyYXZJ/coFaobWBi8Y96Rw5NZZRYQQXLIalC/Dhndm
AHckpstEtx2i8f6yxEUOgPvV/gD7Akn92RpqOGW0g/kYpXjGqZQy9PVHGy61sInY
HSkcOspIkJiS6WyJY9JcvJPM6ns4b84GE9qoUlWVF3RWJk1dqYCw5hz4U8LFyxsF
R6WhYiImvjxBLpab55rSqbGkzjI2z+ucDZyl1gqIv9U6qceVsgRyuqdfVN4deU22
LzO5IEDhnGdFqg9KQY7u8zm686Ejs64T1sh0y4GOmGsSg+P6nsqkdlXH8C+Cf03F
lqPFg8WQC7ojl/S8dPmkT5tcJh3BPwIWuvbtVjFOGQc8x0lb+NwK8h2Nsn6LNazS
0H90adh/IyYX4sBMokrpxAi+gMAWiyJHIHLeH2itNKtAQd3qQowbrWNswJSgJzsT
JuJ7uqRKAFkE6nCeAkuj/6KHHMPsfCAffVdyGaWqhoxmPOrnVgECggEBAOrCCwiC
XxwUgjOfOKx68siFJLfHf4vPo42LZOkAQq5aUmcWHbJVXmoxLYSczyAROopY0wd6
Dx8rqnpO7OtZsdJMeBSHbMVKoBZ77hiCQlrljcj12moFaEAButLCdZFsZW4zF/sx
kWIAaPH9vc4MvHHyvyNoB3yQRdevu57X7xGf9UxWuPil/jvdbt9toaraUT6rUBWU
GYPNKaLFsQzKsFWAzp5RGpASkhuiBJ0Qx3cfLyirjrKqTipe3o3gh/5RSHQ6VAhz
gdUG7WszNWk8FDCL6RTWzPOrbUyJo/wz1kblsL3vhV7ldEKFHeEjsDGroW2VUFlS
asAHNvM4/uYcOSECggEBANYH0427qZtLVuL97htXW9kCAT75xbMwgRskAH4nJDlZ
IggDErmzBhtrHgR+9X09iL47jr7dUcrVNPHzK/WXALFSKzXhkG/yAgmt3r14WgJ6
5y7010LlPFrzaNEyO/S4ISuBLt4cinjJsrFpoo0WI8jXeM5ddG6ncxdurKXMymY7
EOF
>> b64 += <<EOF
gff0GJCOMZ65pMSy3A3cSAtjlKnb4fWzuHD5CFbusN4WhCT/tNxGNSpzvxd8GIDs
nY7exs9L230oCCpedVgcbayHCbkChEfoPzL1e1jXjgCwCTgt8GjeEFqc1gXNEaUn
O8AJ4VlR8fRszHm6yR0ZUBdY7UJddxQiYOzt0S1RLlECggEAbdcs4mZdqf3OjejJ
06oTPs9NRtAJVZlppSi7pmmAyaNpOuKWMoLPElDAQ3Q7VX26LlExLCZoPOVpdqDH
KbdmBEfTR4e11Pn9vYdu9/i6o10U4hpmf4TYKlqk10g1Sj21l8JATj/7Diey8scO
sAI1iftSg3aBSj8W7rxCxSezrENzuqw5D95a/he1cMUTB6XuravqZK5O4eR0vrxR
AvMzXk5OXrUEALUvt84u6m6XZZ0pq5XZxq74s8p/x1JvTwcpJ3jDKNEixlHfdHEZ
ZIu/xpcwD5gRfVGQamdcWvzGHZYLBFO1y5kAtL8kI9tW7WaouWVLmv99AyxdAaCB
Y5mBAQKCAQEAzU7AnorPzYndlOzkxRFtp6MGsvRBsvvqPLCyUFEXrHNV872O7tdO
GmsMZl+q+TJXw7O54FjJJvqSSS1sk68AGRirHop7VQce8U36BmI2ZX6j2SVAgIkI
9m3btCCt5rfiCatn2+Qg6HECmrCsHw6H0RbwaXS4RZUXD/k4X+sslBitOb7K+Y+N
Bacq6QxxjlIqQdKKPs4P2PNHEAey+kEJJGEQ7bTkNxCZ21kgi1Sc5L8U/IGy0BMC
PvJxssLdaWILyp3Ws8Q4RAoC5c0ZP0W2j+5NSbi3jsDFi0Y6/2GRdY1HAZX4twem
Q0NCedq1JNatP1gsb6bcnVHFDEGsj/35oQKCAQEAgmWMuSrojR/fjJzvke6Wvbox
FRnPk+6YRzuYhAP/YPxSRYyB5at++5Q1qr7QWn7NFozFIVFFT8CBU36ktWQ39MGm
cJ5SGyN9nAbbuWA6e+/u059R7QL+6f64xHRAGyLT3gOb1G0N6h7VqFT25q5Tq0rc
Lf/CvLKoudjv+sQ5GKBPT18+zxmwJ8YUWAsXUyrqoFWY/Tvo5yLxaC0W2gh3+Ppi
EDqe4RRJ3VKuKfZxHn5VLxgtBFN96Gy0+Htm5tiMKOZMYAkHiL+vrVZAX0hIEuRZ
EOF
>> der = b64.unpack("m").first
>> c = DerParse.new(der).first_node.first_child
>> version = c.value
=> 0
>> c = c.next_node
>> n = c.value
=> 80071596234464993385068908004931... # (etc)
>> c = c.next_node
>> e = c.value
=> 65537
>> c = c.next_node
>> d = c.value
=> 58438813486895877116761996105770... # (etc)
>> c = c.next_node
>> p = c.value
=> 29635449580247160226960937109864... # (etc)
>> c = c.next_node
>> q = c.value
=> 27018856595256414771163410576410... # (etc)

What I ve done, in case you don t speak Ruby, is take the two chunks of plausible-looking base64 data, chuck them together into a variable named b64, unbase64 it into a variable named der, pass that into a new DerParse instance, and then walk the DER value tree until I got all the values I need. Interestingly, the q value actually traverses the split in the two chunks, which means that there s always the possibility that there are lines missing from the key. However, since p and q are supposed to be prime, we can sanity check them to see if corruption is likely to have occurred:

>> require "openssl"
>> OpenSSL::BN.new(p).prime?
=> true
>> OpenSSL::BN.new(q).prime?
=> true

Excellent! The chances of a corrupted file producing valid-but-incorrect prime numbers isn t huge, so we can be fairly confident that we ve got the real p and q. Now, with the help of another one of my creations we can use e, p, and q to create a fully-operational battle key:

>> require "openssl/pkey/rsa"
>> k = OpenSSL::PKey::RSA.from_factors(p, q, e)
=> #<OpenSSL::PKey::RSA:0x0000559d5903cd38>
>> k.valid?
=> true
>> k.verify(OpenSSL::Digest::SHA256.new, k.sign(OpenSSL::Digest::SHA256.new, "bob"), "bob")
=> true

and there you have it. One fairly redacted-looking private key brought back to life by maths and far too much free time. Sorry Mr. Finn, I hope you re not still using that key on anything Internet-facing.

What About Other Key Types? EC keys are very different beasts, but they have much the same problems as RSA keys. A typical EC key contains both private and public data, and the public portion is twice the size so only about 1/3 of the data in the key is private material. It is quite plausible that you can redact an EC key and leave all the actually private bits exposed.

What Do We Do About It? In short: don t ever try and redact real private keys. For documentation purposes, just put KEY GOES HERE in the appropriate spot, or something like that. Store your secrets somewhere that isn t a public (or even private!) git repo. Generating a dummy private key and sticking it in there isn t a great idea, for different reasons: people have this odd habit of reusing demo keys in real life. There s no need to encourage that sort of thing.

Technically the pieces aren t 100% aligned with the underlying DER, because of how base64 works. I felt it was easier to understand if I stuck to chopping up the base64, rather than decoding into DER and then chopping up the DER.

Dirk Eddelbuettel: #26: Upgrading to R 4.0.0

Welcome to the 26th post in the rationally regularized R revelations series, or R⁴ for short. R 4.0.0 was released two days ago, and a casual glance at some social media conversations appears to suggest quite some confusion, almost certainly some misunderstandings, and possibly also a fair amount of fear, uncertainty, and doubt about the process. So I thought I could show how I upgrade my own main workstation, live and in colour without a safety net. (Almost: I did upgrade my laptop yesterday which went swimmingly, if more slowly.) So here is a fresh video about upgrading to R 4.0.0, with some support slides as usual:

The slides used in the video are at this link. A few quick follow-ups to the live nature of this. The pbdZMQ package did in fact install smoothly once the (Ubuntu) -dev packages for Zero MQ were (re-)installed; then IRkernel also followed. BioConductor completed once I realized that GOSemSim needed the annotation package GO.db to be updated, that allowed MNF to install. So the only bug, really, was the circular depdency between pkgload and testthat. Overall, not bad at all for a quick afternoon session! And as mentioned, if you are interested and have questions concerning use of R on a .deb based system like Debain or Ubuntu (or Mint or ), the r-sig-debian list is a very good and friendly place to ask them. If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Enrico Zini: News and politics links

Theresa Kachindamoto - Wikipedia

empowerment people politics archive.org

2020-03-16

Theresa Kachindamoto is the paramount chief, or Inkosi, of the Dedza District in the central region of Malawi. She has informal authority over more than 900,000 people. She is known for her forceful action in dissolving child marriages and insisting on education for both girls and boys.

KKK Member Walks up to Black Musician in Bar-but It s Not a Joke

empowerment people politics archive.org

2020-03-16

Daryl Davis is no ordinary musician. He s played with President Clinton and tours the country playing burnin boogie woogie piano and sharing musical stylings inspired by greats like Fats Domino, Chuck Berry and Jerry Lee Lewis. He s a highly respected and electrifying performer who is currently an integral member of The Legendary Blues Band (formerly

The dark side of electronic waste recycling

news politics archive.org

2020-03-16

An undercover tracking program is revealing the toll of the e-waste trade.

The dark side of .io: How the U.K. is making web domain profits from a shady Cold War land deal Gigaom

news politics archive.org

2020-03-16

The .io country code top-level domain is pretty popular right now, particularly among tech startups that want to take advantage of the snappy input/output reference and the relative availability of names Fusion.io, Wise.io and Import.io are just a few examples. But who benefits from the sale of .io domains? Sadly, not the people who ultimately should.

US bans shock treatment on children with special needs at Boston-area school

news politics archive.org

2020-03-16

FDA ban brings an end to decades-long battle against use of aversive therapy at the Judge Rotenberg Center in Massachusetts

Julien Danjou: One year of Mergify

It has been close to a year now that I've incorporated my new company, Mergify. I've been busy, and I barely wrote anything about it so far. Now is an excellent time to take a break and reflect a bit on what happened during those last 12 months.

What problem does Mergify solve?Mergify is a powerful automation engine for GitHub pull requests. It allows you to automate everything and especially merging. You write rules, and it handles the rest.
Example of rule matching returned in GitHub checks
For example, let's say you want your pull request to be merged, e.g., once your CI passes and the pull request has been approved. You just write such a rule, and our engine merges the pull request as soon as it's ready.We also deal with more advanced use cases. For instance, we provide a merge queue so your pull requests are merged serially and tested by your CI one after another avoiding any regression in your code.Our goal is to make pull request management and automation easy. You can use your bot to trigger a rebase of your pull requests, or a backport to a different branch, just with a single comment.
Some people like to make bots talk to each other.

A New AdventureMergify is the first company that I ever started. I did run some personal businesses before, created non-profit organizations, built FOSS projects but I never created a company from scratch, even less with an associate.Indeed, I've chosen to build the company with my old friend Mehdi. We've known each others for 7 years now, and have worked together all that time on different open-source projects. Having worked with each other for so long has probably been a critical factor in the success of our venture so far.I had little experience sharing the founding seats with someone, and tons of reading seemed to indicate that it would be a tough ride. Picking the right business partner(s) can be a hard task. Luckily, after working so much time together, Mehdi and I both know our strengths and weaknesses well enough to be able to circumvent them. On the other hand, we both have similar backgrounds as software engineers. That does not help to cover all the hats you need to wear when building a company. Over time, we found arrangements to cover most of those equally between us.We don't have any magical advice to give on this. As in every relationship, communication is the key, and the #1 factor of success.

Getting UsersI don't know if we got lucky, but we got users and customers pretty early. We used a few cooperative projects as guinea pigs first, and they were brave enough to try our service and give us feedback. No repository has been harmed during this first phase!Then, as soon as we managed to get our application on the GitHub Marketplace, we saw a steady number of users coming to us.This has been fantastic as it allowed us to get feedback rapidly. We set up a form asking users for feedback after they used Mergify for a couple of weeks. What we hear is that users were happy, that the documentation was confusing and that some features were buggy or missing. We planned all of those ideas as our future work in our roadmap, using the principles we described a few months ago.
How we handle our roadmap for Mergify
Whatever you re building a company, a product, or a house a time comes where you need planning. Pushing random buttons to move forward does not work anymore. You need to take a step back to
Julien DanjouMergify
If you're curious, you can read this article.
We tried various strategies to get new users, but so far, organic growth has been our #1 way of onboarding new users. Like many small startups out there, we're not that good at marketing and executing strategies.We provide our service for free for open-source projects We are now powering many organizations, such as Mozilla, Amazon Web Services, Ceph and Fedora.

Working with GitHubWorking with GitHub has been complicated. When you build an application for a marketplace, your business is entirely dependent on the platform you develop for both in terms of features and quality of service.In our case, we hit quite many bugs with GitHub. Their support has mostly been fast to answer, but some significant issues are still opened months later. The truth is that the GitHub API could deserve more love and care from GitHub. For example, their GraphQL API is a work in progress for years and miss out many essential features.
GitHub service status is not always green.
We dealt and still deal with all those issues. It obviously impacts our operations and decreases our overall velocity. We regularly have to find new ways to sidestep GitHub limitations.You have no idea how we wished for GitHub to be open-source. The idea of not having access to their code and understand how it works is so frustrating that we publish our engine as an open-source project. That allows all of our users to see how it works and even propose enhancements.

Automate all the wayWe're a tiny startup, and we decided to bootstrap our company. We never took any funding. From the beginning, it has been clear to us that we had to think and act like we had no resources. We're built around a scarcity mindset. Every decision we make is based on the assumption that we basically are very limited in terms of money and time.We basically act like any wrong choice we do could (virtually) kill the company. We only do what is essential, we ship fast, and we automate everything.For example, we have built our whole operation about CI/CD systems, and pushing any new fix or feature in production is done in a matter of minutes. It's not uncommon for us to push a fix from our phone, just by reviewing some code or editing a file.

GrowthWe're extremely happy with our steady growth and more users using our service. We now manage close to 30k repositories and merge 15k pull requests per month for our users.That's a lot of mouse clicks saved!If you want to try Mergify yourself, it's a single click log-in using your GitHub account. Check it out!

Jonathan Dowland: Coil

Peter Christopherson and Jhonn Balance, from Santa Sangre

A friend asked me to suggest five tracks by Coil that gave an introduction to their work. Trying to summarize Coil in 5 tracks is tough. I think it's probably impossible to fairly summarize Coil with any subset of their music, for two reasons. Firstly, their music was the output of their work but I don't think is really the whole of the work itself. There's a real mystique around them. They were deeply interested in arcania, old magic, Aleister Crowley, scatology; they were both openly and happily gay and their work sometimes explored their experiences in various related underground scenes and sub-cultures; they lost friends to HIV/AIDS and that had a profound impact on them. They had a big influence on some people who discovered them who were exploring their own sexualities at the time and might have felt excluded from mainstream society. They frequently explored drugs, meditation and other ways to try to expand and open their minds; occultism. They were also fiercely anti-commercial, their stuff was released in limited quantities across a multitude of different music labels, often under different names, and often paired with odd physical objects, runes, vials of blood, etc. Later fascinations included paganism and moon worship. I read somewhere that they literally cursed one of their albums. Secondly, part of their "signature" was the lack of any consistency in their work, or to put it another way, their style over time varied enormously. I'm also not necessarily well-versed in all their stuff, I'm part way on this journey myself... but these are tracks which stand out at least from the subset I've listened to. Both original/core members of Coil have passed away and the legal status of their catalogue is in a state of limbo. Some of these songs are available on currently-in-print releases, but all such releases are under dispute by some associate or other.

1. Heaven's Blade Like (probably) a lot of Coil songs, this one exists in multiple forms, with some dispute about which are canonical, which are officially sanctioned, etc. the video linked above actually contains 5 different versions, but I've linked to a time offset to the 4th: "Heaven's Blade (Backwards)". This version was the last to come to light with the recent release of "Backwards", an album originally prepared in the 90s at Trent Reznor's Nothing Studios in New Orleans, but not finished or released. The circumstances around its present-day release, as well as who did what to it and what manipulation may have been performed to the audio a long time after the two core members had passed, is a current topic in fan circles. Despite that, this is my preferred version. You can choose to investigate the others, or not, at your own discretion.

2. how to destroy angels (ritual music for the accumulation of male sexual energy) A few years ago, "guidopaparazzi", a user at the Echoing the Sound music message board attempted to listen to every Coil release ever made and document the process. He didn't do it chronologically, leaving the EPs until near the end, which is when he tackled this one (which was the first release by Coil, and was the inspiration behind the naming of Trent Reznor's one-time side project "How To Destroy Angels"). Guido seemed to think this was some kind of elaborate joke. Personally I think it's a serious piece and there's something to it but this just goes to show, different people can take things in entirely different ways. Here's Guido's review, and you can find the rest of his reviews linked from that one if you wish. https://archive.org/details/Coil-HowToDestroyAngels1984

3. Red Birds Will Fly Out Of The East And Destroy Paris In A Night Both "Musick To Play In The Dark" volumes (one and two) are generally regarded as amongst the most accessible entry points to the Coil discography. This is my choice of cut from volume 1. For some reason this reminds me a little of some of the background music from the game "Unreal Tournament". I haven't played that in at least 15 years. I should go back and see if I can figure out why it does. The whole EP is worth a listen, especially at night. https://archive.org/details/CoilMusickToPlayInTheDarkVol1/Coil+-+Musick+To+Play+In+The+Dark+Vol+1+-+2+Red+Birds+Will+Fly+Out+Of+The+East+And+Destroy+Paris+In+A+Night.flac

4. Things Happen It's tricky to pick a track from either "Love's Secret Domain" or "Horse Rotorvator"; there are other choices which I think are better known and loved than this one but it's one that haunted me after I first heard it for one reason or another, so here it is.

5. The Anal Staircase Track 1 from Horse Rotorvator. What the heck is a Horse Rotorvator anyway? I think it was supposed to have been a lucid nightmare experienced by the vocalist Jhonn Balance. So here they wrote a song about anal sex. No messing about, no allusion particularly, but why should there be? https://archive.org/details/CoilHorseRotorvator2001Remaster/Coil+-+Horse+Rotorvator+%5B2001+remaster%5D+-+01+The+Anal+Staircase.flac

Bonus 6th: 7-Methoxy-B-Carboline (Telepathine) From the drone album "Time Machines", which has just been re-issued by DIAS records, who describe it as "authorized". Each track is titled by the specific combination of compounds that inspired its composition, supposedly. Or, perhaps it's a "recommended dosing" for listening along. https://archive.org/details/TimeMachines-TimeMachines

Post-script If those piqued your interest, there's some decent words and a list of album suggestions in this Vinyl Factory article. Finally, if you can track them down, Stuart Maconie had two radio shows about Coil on his "Freak Zone" programme. The main show discusses the release of "Backwards", including an interview with collaborator Danny Hyde, who was the main person behind the recent re-issue. The shorter show is entitled John Doran uncoils Coil. Guest John Doran from The Quietus discusses the group and their history interspersed with Coil tracks and tracks from their contemporaries. Interestingly they chose a completely different set of 5 tracks to me.

Matthias Klumpp: Adding fonts to software centers

L

ast year, the AppStream specification gained proper support for adding metadata for fonts, after Richard Hughes did some work on it years ago. We weren t happy with how fonts were handled at that time, so we searched for better solutions, which is why this took a bit longer to be done. Last year, I was implementing the final support for fonts in both appstream-generator (the metadata extractor used by Debian and a few others) as well as the AppStream specification. This blogpost was sitting on my todo list as a draft for a long time now, and I only just now managed to finish it, so sorry for announcing this so late. Fonts are already available via AppStream for a year, and this post just sums up the status quo and some neat tricks if you want to write metainfo files for fonts. If you are following AppStream (or the Debian fonts list), you know everything already

. Both Richard and I first tried to extract all the metadata to display fonts in a proper way to the users from the font files directly. This turned out to be very difficult, since font metadata is often wrong or incomplete, and certain desirable bits of metadata (like a longer description) are missing entirely. After messing around with different ways to solve this for days (afterall, by extracting the data from font files directly we would have hundreds of fonts directly available in software centers), I also came to the same conclusion as Richard: The best and easiest solution here is to mandate the availability of metainfo files per font. Which brings me to the second issue: What is a font? For any person knowing about fonts, they will understand one font as one font face, e.g. Lato Regular Italic or Lato Bold . A user however will see the font family as a font, e.g. just Lato instead of all the font faces separated out. Since AppStream data is used primarily by software centers, we want something that is easy for users to understand. Hence, an AppStream font components really describes a font family or collection of fonts, instead of individual font faces. We do also want AppStream data to be useful for system components looking for a specific font, which is why font components will advertise the individual font face names they contain via a

<provides/>

-tag. Naming fonts and making them identifiable is a whole other issue, I used a document from Adobe on font naming issues as a rough guideline while working on this. How to write a good metainfo file for a font is best shown with an example. Lato is a well-looking font family that we want displayed in a software center. So, we write a metainfo file for it an place it in

/usr/share/metainfo/com.latofonts.Lato.metainfo.xml

for the AppStream metadata generator to pick up:

<?xml version="1.0" encoding="UTF-8"?>
<component type="font">
  <id>com.latofonts.Lato</id>
  <metadata_license>FSFAP</metadata_license>
  <project_license>OFL-1.1</project_license>
  <name>Lato</name>
  <summary>A sanserif type face fam ily</summary>
  <description>
    <p>
      Lato is a sanserif type face fam ily designed in the Sum mer 2010 by Warsaw-based designer
       ukasz Dziedzic ( Lato  means  Sum mer  in Pol ish). In Decem ber 2010 the Lato fam ily
      was pub lished under the open-source Open Font License by his foundry tyPoland, with
      sup port from Google.
    </p>
  </description>
  <url type="homepage">http://www.latofonts.com/</url>
  <provides>
    <font>Lato Regular</font>
    <font>Lato Black Italic</font>
    <font>Lato Black</font>
    <font>Lato Bold Italic</font>
    <font>Lato Bold</font>
    <font>Lato Hairline Italic</font>
    ...
  </provides>
</component>

When the file is processed, we know that we need to look for fonts in the package it is contained in. So, the appstream-generator will load all the fonts in the package and render example texts for them as an image, so we can show users a preview of the font. It will also use heuristics to render an icon for the respective font component using its regular typeface. Of course that is not ideal what if there are multiple font faces in a package? What if the heuristics fail to detect the right font face to display? This behavior can be influenced by adding

<font/>

tags to a

<provides/>

tag in the metainfo file. The font-provides tags should contain the fullnames of the font faces you want to associate with this font component. If the font file does not define a fullname, the family and style are used instead. That way, someone writing the metainfo file can control which fonts belong to the described component. The metadata generator will also pick the first mentioned font name in the

<provides/>

list as the one to render the example icon for. It will also sort the example text images in the same order as the fonts are listed in the provides-tag. The example lines of text are written in a language matching the font using Pango. But what about symbolic fonts? Or fonts where any heuristic fails? At the moment, we see ugly tofu characters or boxes instead of an actual, useful representation of the font. This brings me to an inofficial extension to font metainfo files, that, as far as I know, only appstream-generator supports at the moment. I am not happy enough with this solution to add it to the real specification, but it serves as a good method to fix up the edge cases where we can not render good example images for fonts. AppStream-Generator supports the FontIconText and FontSampleText custom AppStream properties to allow metainfo file authors to override the default texts and autodetected values. FontIconText will override the characters used to render the icon, while FontSampleText can be a line of text used to render the example images. This is especially useful for symbolic fonts, where the heuristics usually fail and we do not know which glyphs would be representative for a font. For example, a font with mathematical symbols might want to add the following to its metainfo file:

<custom>
  <value key="FontIconText"> </value>
  <value key="FontSampleText">       ...         </value>
</custom>

Any unicode glyphs are allowed, but asgen will but some length restrictions on the texts. So, In summary:

Fonts are hard
I need to blog faster
Please add metainfo files to your fonts and submit them upstream if you can!
Fonts must have a metainfo file in order to show up in GNOME Software, KDE Discover, AppCenter, etc.
The new font specification is backwards compatible to Richard s pioneer work in 2014
The appstream-generator supports a few non-standard values to influence how font images are rendered that you might be interested in (maybe we can do something like that for appstream-builder as well)
The appstream-generator does not (yet?) support the <extends/> logic Richard outlined in his blog post, mainly because it wasn t necessary in Debian/Ubuntu/Arch yet (which is asgen s primary audience), and upstream projects would rarely want to write multiple metainfo files.
The metaInfo files are not supposed to replace the existing fontconfig files, and we can not generate them from existing metadata, sadly
If you want a more detailed look at writing font metainfo files, take a look at the AppStream specification.
Please write more font metadata

Adnan Hodzic: Secure traffic to ZNC on Synology with Let s Encrypt

I ve been using IRC since late 1990 s, and I continue to do so to this day due to it (still) being one of the driving development forces in various open source communities. Especially in Linux development and some of my acquintances I can only get in touch with via IRC :) My Setup On my Synology NAS I run ZNC (IRC bouncer/proxy) to which I connect using various IRC clients (irssi/XChat Azure/AndChat) from various platforms (Linux/Mac/Android). In this case ZNC serves as a gateway and no matter which device/client I connect from, I m always connected to same IRC servers/chat rooms/settings when I left off. This is all fine and dandy, but connecting from external networks to ZNC means you will hand in your ZNC credentials in plain text. Which is a problem for me, even thought we re only talking about IRC bouncer/proxy. With that said, how do we encrypt external traffic to our ZNC? HowTo: Chat securely with ZNC on Synology using Let s Encrypt SSL certificate For reference or more thorough explanation of some of the steps/topics please refer to: Secure (HTTPS) public access to Synology NAS using Let s Encrypt (free) SSL certificate

Requirements:

Synology NAS running DSM >= 6.0
Sub/domain name with ability to update DNS records
SSH access to your Synology NAS

1: DNS setup Create A record for sub/domain you d like to use to connect to your ZNC and point it to your Synology NAS external (WAN) IP. For your reference, subdomain I ll use is: irc.hodzic.org 2: Create Let s Encrypt certificate

DSM: Control Panel > Security > Certificates > Add

Followed by:

Add a new certificate > Get a certificate from Let's Encrypt

Followed by adding domain name A record was created for in Step 1, i.e:

Get a certificate from Let's Encrypt for irc.hodzic.org

After certificate is created, don t forget to configure newly created certificate to point to correct domain name, i.e:

3: Install ZNC In case you already have ZNC installed, I suggest you remove it and do a clean install. Mainly due to some problems with package in past, where ZNC wouldn t start automatically on boot which lead to creating projects like: synology-znc-autostart. In latest version, all of these problems have been fixed and couple of new features have been added. ZNC can be installed using Synology s Package Center, if community package sources are enabled. Which can simply be done by adding new package sources:

Name: SynoCommunity
Location: http://packages.synocommunity.com

Enable Community package sources in Synology Package Center

To successfuly authenticate newly added source, under General tab, Trust Level should be set to Any publisher As part of installation process, ZNC config will be run with most sane/useful options and admin user will be created allowing you access to ZNC webadmin. 4: Secure access to ZNC webadmin Now we want to bind our sub/domain created in Step 1 to ZNC webadmin, and secure external access to it. This can be done by creating a reverse proxy. As part of this, you need to know which port has been allocated for SSL in ZNC Webadmin, i.e:

ZNC Webadmin > Global settings - Listen Ports

In this case, we can see it s 8251. Reverse Proxy can be created in:

DSM: Control Panel > Application Portal > Reverse Proxy > Create

Where sub/domain created in Step 1 needs to be point to ZNC SSL port on localhost, i.e:

ZNC Webadmin is now available via HTTPS on external network for the sub/domain you setup as part of Step 1, or in my case:

As part of this step, in ZNC webadmin I d advise you to create IRC servers and chatrooms you would like to connect to later. Step 5: Create .pem file from LetsEncrpyt certificate for ZNC to use On Synology, Let s Encrypt certificates are stored and located on:

/usr/syno/etc/certificate/_archive/

In case you have multiple certificates, based on date your certificate was created, you can determine in which directory is your newly generated certificated stored, i.e:

drwx------ 2 root root 4096 Sep 10 12:57 JeRh3Y

Once it s determined which certifiate is the one we want use, generate .pem by running following: sudo cat /usr/syno/etc/certificate/_archive/JeRh3Y/ privkey,cert,chain .pem > /usr/local/znc/var/znc.pem After this restart ZNC: sudo /var/packages/znc/scripts/start-stop-status stop && sudo /var/packages/znc/scripts/start-stop-status start 6: Configure IRC client In this example I ll use XChat Azure on MacOS, and same procedure should be identical for HexChat/XChat clients on any other platform. Altough all information is picked up from ZNC itself, user details will need to be filled in. In my setup I automatically connect to freenode and oftc networks, so I created two for local network and two for external network usage, later is the one we re concentrating on. On General tab of our newly created server, hostname for our server should be the sub/domain we ve setup as part of Step 1 , and port number should be the one we defined in Step 4 , SSL checkbox must be checked.

On Connecting tab Server password field needs to be filled in following format:

johndoe/freenode:password

Where, johndoe is ZNC username. freenode is ZNC network name, and password is ZNC password.

Xchat Azure: Network list - Connecting tab

freenode in this case must first be created as part of ZNC webadmin configuration, mentioned in step 4 . Same case is for oftc network configuration. As part of establishing the connection, information about our Let s Encrypt certificate will be displayed, after which connection will be established and you ll be automatically logged into all chatrooms. Happy hacking!

Chris Lamb: Free software activities in August 2017

Here is my monthly update covering what I have been doing in the free software world in August 2017 (previous month):

Created ZeroCoolOS, a live operating system that plays the film Hackers (1995) on a continuous loop.
Sent a patch for pristine-tar to allow storage of detached upstream signatures. (#871809)
Worked more on Lintian, a static analysis tool for Debian packages, reporting on various errors, omissions and quality-assurance issues to the maintainer (previous changes):
- Fix an apache2-unparsable-dependency false positive by allowing periods in dependency names. (#873701)
- Ignore "repacked" packages when checking for upstream source tarball signatures as they will never match.
- Downgrade the severity of orig-tarball-missing-upstream-signature. (#870722)
- From a suggestion by Theodore Ts'o, expand the explanation of orig-tarball-missing-upstream-signature to include the location of where dpkg-source looks.
- Address a number of issues in the copyright-year-in-future tag including preventing false positives in port numbers, email addresses, ISO standard numbers and street addresses (#869788), as well as "meta" or testing statements (#873323). In addition, report all violating years in a line and expand the testsuite.
- Don't match quoted "FIXME" variants of file-contains-fixme-placeholder (#870199), avoid checking copyright_hints files (#872843) and downgrade the tag's severity.
- Apply a patch from Alex Muntada to recommend "substr" over of "substring" in mentions-deprecated-usr-lib-perl5-directory. (#871767)
- Prevent missing-build-dependency-for-dh_-command false positives exposed by following the advice in useless-autoreconf-build-depends. (#869541)
- Ensure readme-debian-contains-debmake-template also checks for files containing "Automatically generated by debmake".
- Check python3-foo packages have a Section: python, not just python2-foo. (#870272)
- Check for packages shipping compiled Java class files. (#873211)
- Additionally consider .cljc files to avoid codeless-jar warnings. (#870649)
- Prevent desktop-entry-lacks-keywords-entry false positives for Link and Directory-style .desktop files. (#873702)
- Split out Python checks from checks/scripts.pm check to a new, source check of type source.
- Check for python-foo without a corresponding python3-foo package. (#870681)
- Complain about packages that Build-Depend on python-sphinx only. (#870730)
- Warn about packages that alternatively Build-Depend on the Python 2 and Python 3 versions of Sphinx. (#870758)
- Check for packages that depend on Python 2.x. (#870822)
- Correct false positives in unconditional-use-of-dpkg-statoverride by detecting "if !" as a shell prefix. (#869587)
- Alert on for missing calls to dpkg-maintscript-helper(1) in maintainer scripts. (#872042)
- Check for packages using sensible-utils without declaring a dependency after splitting from debianutils. (#872611)
- Warn about scripts using nodejs as an interpreter now that the nodejs script provides /usr/bin/node. (#873096)
- Remove recommendations to add a Testsuite: autopkgtest field to debian/control and emit a new tag the package if it does so. (#865531)
- Recognise autopkgtest-pkg-elpa as a valid test suite. (#873458)
- Add note to /etc/bash_completion.d's obsolete path warning output regarding stricter filename requirements. (#814599)
- Add 4.0.1 and 4.1.0 as known Policy standards versions.
- Apply a patch from Maia Everett to avoid British spellings under the en_US locale. (#868897)
- Stop emitting maintainer,uploader -address-causes-mail-loops for @packages.debian.org addresses. (#871575)
- Modify Lintian::Data's all subroutine to always return keys in insertion order.
- Apply a patch from Steve Langasek to accomodate binutils outputting symbols in a different format on the ppc64el architecture. (#869750)
- Add an explicit test for packages including external fonts via the Google Font and TypeKit APIs. (#873434)
- Add missing entries in internal Test-For fields to make development/testing workflow less error-prone.
Sent three pull requests to git-buildpackage, a tool to assist in Debian packaging from Git repositories:
- Make pq --abbrev= configurable. (#872351)
- Use build profiles to avoid installation of test dependencies. (#31)
- Correct "allow to" grammar. (#30)
Updated travis.debian.net (my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform for testing):
- Move away from deb.debian.org; Travis appears to be using a HTTP proxy that strips SRV records. (commit)
- Highlight double quotes are required for TRAVIS_DEBIAN_EXTRA_REPOSITORY. (commit)
- Use force-unsafe-io. (commit)
- Clarify docs when upstream already has a travis.yml file. (#46)
- Make documentation easier to copy-paste. (commit)
Merged a pull request in django-slack, my library to easily post messages to the Slack group-messaging utility, where instantiation of a SlackException was failing. (#71)
Assigned two pull requests to the Redis key-value database store to correct "did not received" and "faield" typos. (#4216 & #4215).

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced either maliciously or accidentally during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area. This month I:

Presented a status update at Debconf17 in Montr al, Canada alongside Holger Levsen, Maria Glukhova, Steven Chamberlain, Vagrant Cascadian, Valerie Young and Ximin Luo.
I worked on the following issues upstream:
- glib2.0: Please make the output of gio-querymodules reproducible. (...)
- gcab: Please make the output reproducible. (...)
- gtk+2.0: Please make the immodules.cache files reproducible. (...)
- desktop-file-utils: Please make the output reproducible. (...)
Within Debian:
- My work as Debian Project Leader (DPL) is covered in my monthly Bits from the DPL email to debian-devel-announce.
- Created isdebianreproducibleyet.com.
- Sent a patch to dpkg to sort the "unused substitution" warnings. (#870221)
- Added a script to devscripts to report on reproducibility status of installed packages. (#872514)
- Modified the Debian archive tools (dak) to automatically reject packages which do not bump their date in debian/changelog. (debian-devel post)
- Fixed an QA issue in snappy that was caught by the reproducible builds continuous integration framework. (#872226)
- I submitted three patches to fix specific reproducibility issues in grap, isa-support & python-numpy.
- Finally, I also performed two non-maintainer uploads (NMUs) for jsmath-fonts (#792319) and xvier (#777330) to make their builds reproducible.
Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
Worked on publishing our weekly reports. (#118, #119, #120, #121 & #122)

I also made the following changes to our tooling:

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

Use name attribute over path to avoid leaking comparison full path in output. (commit)
Add missing skip_unless_module_exists import. (commit)
Tidy diffoscope.progress and the XML comparator (commit, commit)

disorderfs

disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues.

Add a simple autopkgtest smoke test. (commit)

Debian

Patches contributed

openssh: Quote the IP address in ssh-keygen -f suggestions. (#872643)
libgfshare:
- SIGSEGV if /dev/urandom is not accessible. (#873047)
- Add bindnow hardening. (#872740)
- Support nodoc build profile. (#872739)
devscripts:
- Enable hardening buildflags for /usr/bin/debpkg. (#873379)
- Add missing scripts/debc to .gitignore. (#873381)
memcached: Add hardening to systemd .service file. (#871610)
googler: Tidy long and short package descriptions. (#872461)
gnome-split: Homepage points to domain-parked website. (#873037)

Uploads

python-django 1:1.11.4-1 New upstream release.
redis:
- 4:4.0.1-3 Drop yet more non-deterministic tests.
- 4:4.0.1-4 Tighten systemd/seccomp hardening.
- 4:4.0.1-5 Drop even more tests with timing issues.
- 4:4.0.1-6 Don't install completions to /usr/share/bash-completion/completions/debian/bash_completion/.
- 4:4.0.1-7 Don't let sentinel integration tests fail the build as they use too many timers to be meaningful. (#872075)
python-gflags 1.5.1-3 If SOURCE_DATE_EPOCH is set, either use that as a source of current dates or the UTC-version of the file's modification time (#836004), don't call update-alternatives --remove in postrm. update debian/watch/Homepage & refresh/tidy the packaging.
bfs 1.1.1-1 New upstream release, tidy autopkgtest & patches, organising the latter with Pq-Topic.
python-daiquiri 1.2.2-1 New upstream release, tidy autopkgtests & update travis.yml from travis.debian.net.
aptfs 2:0.10-2 Add upstream signing key, refer to /usr/share/common-licenses/GPL-3 in debian/copyright & tidy autopkgtests.
adminer 4.3.1-2 Add a simple autopkgtest & don't install the Selenium-based tests in the binary package.
zoneminder (1.30.4+dfsg-2) Prevent build failures with GCC 7 (#853717) & correct example /etc/fstab entries in README.Debian (#858673).

Finally, I reviewed and sponsored uploads of astral, inflection, more-itertools, trollius-redis & wolfssl.

Debian LTS

This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:

"Frontdesk" duties, triaging CVEs, etc.
Issued DLA 1049-1 for libsndfile preventing a remote denial of service attack.
Issued DLA 1052-1 against subversion to correct an arbitrary code execution vulnerability.
Issued DLA 1054-1 for the libgxps XML Paper Specification library to prevent a remote denial of service attack.
Issued DLA 1056-1 for cvs to prevent a command injection vulnerability.
Issued DLA 1059-1 for the strongswan VPN software to close a denial of service attack.

Debian bugs filed

wget: Please hash the hostname in ~/.wget-hsts files. (#870813)
debian-policy: Clarify whether mailing lists in Maintainers/Uploaders may be moderated. (#871534)
git-buildpackage: "pq export" discards text within square brackets. (#872354)
qa.debian.org: Escape HTML in debcheck before outputting. (#872646)
pristine-tar: Enable multithreaded compression in pristine-xz. (#873229)
tryton-meta: Please combine tryton-modules-* into a single source package with multiple binaries. (#873042)
azure-cli:
- New upstream release. (#871418)
- debian/watch file broken. (#871417)
fwupd-tests: Don't ship test files to generic /usr/share/installed-tests dir. (#872458)
libvorbis: Maintainer fields points to a moderated mailing list. (#871258)
rmlint-gui: Ship a rmlint-gui binary. (#872162)
template-glib: debian/copyright references online source without quotation. (#873619)

FTP Team

As a Debian FTP assistant I ACCEPTed 147 packages: abiword, adacgi, adasockets, ahven, animal-sniffer, astral, astroidmail, at-at-clojure, audacious, backdoor-factory, bdfproxy, binutils, blag-fortune, bluez-qt, cheshire-clojure, core-match-clojure, core-memoize-clojure, cypari2, data-priority-map-clojure, debian-edu, debian-multimedia, deepin-gettext-tools, dehydrated-hook-ddns-tsig, diceware, dtksettings, emacs-ivy, farbfeld, gcc-7-cross-ports, git-lfs, glewlwyd, gnome-recipes, gnome-shell-extension-tilix-dropdown, gnupg2, golang-github-aliyun-aliyun-oss-go-sdk, golang-github-approvals-go-approval-tests, golang-github-cheekybits-is, golang-github-chzyer-readline, golang-github-denverdino-aliyungo, golang-github-glendc-gopher-json, golang-github-gophercloud-gophercloud, golang-github-hashicorp-go-rootcerts, golang-github-matryer-try, golang-github-opentracing-contrib-go-stdlib, golang-github-opentracing-opentracing-go, golang-github-tdewolff-buffer, golang-github-tdewolff-minify, golang-github-tdewolff-parse, golang-github-tdewolff-strconv, golang-github-tdewolff-test, golang-gopkg-go-playground-validator.v8, gprbuild, gsl, gtts, hunspell-dz, hyperlink, importmagic, inflection, insighttoolkit4, isa-support, jaraco.itertools, java-classpath-clojure, java-jmx-clojure, jellyfish1, lazymap-clojure, libblockdev, libbytesize, libconfig-zomg-perl, libdazzle, libglvnd, libjs-emojify, libjwt, libmysofa, libundead, linux, lua-mode, math-combinatorics-clojure, math-numeric-tower-clojure, mediagoblin, medley-clojure, more-itertools, mozjs52, openssh-ssh1, org-mode, oysttyer, pcscada, pgsphere, poppler, puppetdb, py3status, pycryptodome, pysha3, python-cliapp, python-coloredlogs, python-consul, python-deprecation, python-django-celery-results, python-dropbox, python-fswrap, python-hbmqtt, python-intbitset, python-meshio, python-parameterized, python-pgpy, python-py-zipkin, python-pymeasure, python-thriftpy, python-tinyrpc, python-udatetime, python-wither, python-xapp, pythonqt, r-cran-bit, r-cran-bit64, r-cran-blob, r-cran-lmertest, r-cran-quantmod, r-cran-ttr, racket-mode, restorecond, rss-bridge, ruby-declarative, ruby-declarative-option, ruby-errbase, ruby-google-api-client, ruby-rash-alt, ruby-representable, ruby-test-xml, ruby-uber, sambamba, semodule-utils, shimdandy, sjacket-clojure, soapysdr, stencil-clojure, swath, template-glib, tools-analyzer-jvm-clojure, tools-namespace-clojure, uim, util-linux, vim-airline, vim-airline-themes, volume-key, wget2, xchat, xfce4-eyes-plugin & xorg-gtest. I additionally filed 6 RC bugs against packages that had incomplete debian/copyright files against: gnome-recipes, golang-1.9, libdazzle, poppler, python-py-zipkin & template-glib.

Carl Chenet: Send scheduled messages to both Twitter and Mastodon with the Remindr bot

Do you need to send messages to both Twitter and Mastodon? Use the Remindr bot! Remindr is written in Python, released under the GPLv3 license.

1. How to install Remindr Install Remindr from PyPI:

# pip3 install remindr

2. How to configure Remindr First, start by writing a messages.txt file with the following content:

o en Send scheduled messages to both Twitter and Mastodon with the Remindr bot https://carlchenet.com/send-scheduled-messages-to-both-twitter-and-mastodon-with-the-remindr-bot #remindr #twitter #mastodon
x en Follow Carl Chenet for great news about Free Software! https://carlchenet.com #freesoftware

The first field only indicates if the line is the next one to be considered by Remindr, the o indicates the next line to be sent, x means it won t. The second field is the 2-letters code language your content is using, in my example en or fr. Next content on the line will compose the body of your messages to Mastodon and Twitter. You need to configure the Mastodon and the Twitter credentials in order to allow Remindr to send the messages. First you need to generate the credentials. For Twitter, you need to manually create an app on apps.twitter.com. For Mastodon, just launch the following command:

$ register_remindr_app

Some information will be asked by the command. At the end, two files are created, remindr_usercred.txt and remindr_clientcred.txt. You re going to need them for the Remindr configuration above. For the Remindr configuration, here is a complete configuration using the

[mastodon]
instance_url=https://mastodon.social
user_credentials=remindr_usercred.txt
client_credentials=remindr_clientcred.txt
[twitter]
consumer_key=a6lv2gZxkvk6UbQ30N4vFmlwP
consumer_secret=j4VxM2slv0Ud4rbgZeGbBzPG1zoauBGLiUMOX0MGF6nsjcyn4a
access_token=1234567897-Npq5fYybhacYxnTqb42Kbb3A0bKgmB3wm2hGczB
access_token_secret=HU1sjUif010DkcQ3SmUAdObAST14dZvZpuuWxGAV0xFnC
[image]
path_to_image=/home/chaica/blog-carl-chenet.png
[entrylist]
path_to_list=/etc/remindr/messages.txt

Your configuration is complete! Now we have to check if everything is fine.

Read the full documentation on Readthedocs.

3. How to use Remindr Now let s try your configuration by launching Remindr the first time by-hand:

$ remindr -c /etc/remindr/remindr.ini

The messages should appear on both Twitter and Mastodon. 4. How to schedule the Remindr execution The easiest way is to use you user crontab. Just add the following line in your crontab file, editing it with crontab -e

00 10 * * * remindr -c /etc/remindr/remindr.ini

From now on, your message will be sent every day at 10:00AM. Going further with Remindr

Remindr on Gitlab.com (would be awesome starring it if you like/use it)
The official documentation on Readthedocs.io

and finally You can help me developing tools for Mastodon and other social networks by donating anything through Liberaypay (also possible with cryptocurrencies). Any contribution will be appreciated. That s a big factor motivation

Donate You also may follow my account @carlchenet on Mastodon

Norbert Preining: TeX Live 2017 hits Debian/unstable

Yesterday I uploaded the first packages of TeX Live 2017 to Debian/unstable, meaning that the new release cycle has started. Debian/stretch was released over the weekend, and this opened up unstable for new developments. The upload comprised the following packages: asymptote, cm-super, context, context-modules, texlive-base, texlive-bin, texlive-extra, texlive-extra, texlive-lang, texworks, xindy.

I mentioned already in a previous post the following changes:

several packages have been merged, some are dropped (eg. texlive-htmlxml) and one new package (texlive-plain-generic) has been added
luatex got updated to 1.0.4, and is now considered stable
updmap and fmtutil now require either -sys or -user
tlmgr got a shell mode (interactive/scripting interface) and a new feature to add arbitrary TEXMF trees (conf auxtrees)

The last two changes are described together with other news (easy TEXMF tree management) in the TeX Live release post. These changes more or less sum up the new infra structure developments in TeX Live 2017. Since the last release to unstable (which happened in 2017-01-23) about half a year of package updates have accumulated, below is an approximate list of updates (not split into new/updated, though). Enjoy the brave new world of TeX Live 2017, and please report bugs to the BTS! Updated/new packages:
academicons, achemso, acmart, acro, actuarialangle, actuarialsymbol, adobemapping, alkalami, amiri, animate, aomart, apa6, apxproof, arabluatex, archaeologie, arsclassica, autoaligne, autobreak, autosp, axodraw2, babel, babel-azerbaijani, babel-english, babel-french, babel-indonesian, babel-japanese, babel-malay, babel-ukrainian, bangorexam, baskervaldx, baskervillef, bchart, beamer, beamerswitch, bgteubner, biblatex-abnt, biblatex-anonymous, biblatex-archaeology, biblatex-arthistory-bonn, biblatex-bookinother, biblatex-caspervector, biblatex-cheatsheet, biblatex-chem, biblatex-chicago, biblatex-claves, biblatex-enc, biblatex-fiwi, biblatex-gb7714-2015, biblatex-gost, biblatex-ieee, biblatex-iso690, biblatex-manuscripts-philology, biblatex-morenames, biblatex-nature, biblatex-opcit-booktitle, biblatex-oxref, biblatex-philosophy, biblatex-publist, biblatex-shortfields, biblatex-subseries, bibtexperllibs, bidi, biochemistry-colors, bookcover, boondox, bredzenie, breqn, bxbase, bxcalc, bxdvidriver, bxjalipsum, bxjaprnind, bxjscls, bxnewfont, bxorigcapt, bxpapersize, bxpdfver, cabin, callouts, chemfig, chemformula, chemmacros, chemschemex, childdoc, circuitikz, cje, cjhebrew, cjk-gs-integrate, cmpj, cochineal, combofont, context, conv-xkv, correctmathalign, covington, cquthesis, crimson, crossrefware, csbulletin, csplain, csquotes, css-colors, cstldoc, ctex, currency, cweb, datetime2-french, datetime2-german, datetime2-romanian, datetime2-ukrainian, dehyph-exptl, disser, docsurvey, dox, draftfigure, drawmatrix, dtk, dviinfox, easyformat, ebproof, elements, endheads, enotez, eqnalign, erewhon, eulerpx, expex, exsheets, factura, facture, fancyhdr, fbb, fei, fetamont, fibeamer, fithesis, fixme, fmtcount, fnspe, fontmfizz, fontools, fonts-churchslavonic, fontspec, footnotehyper, forest, gandhi, genealogytree, glossaries, glossaries-extra, gofonts, gotoh, graphics, graphics-def, graphics-pln, grayhints, gregoriotex, gtrlib-largetrees, gzt, halloweenmath, handout, hang, heuristica, hlist, hobby, hvfloat, hyperref, hyperxmp, ifptex, ijsra, japanese-otf-uptex, jlreq, jmlr, jsclasses, jslectureplanner, karnaugh-map, keyfloat, knowledge, komacv, koma-script, kotex-oblivoir, l3, l3build, ladder, langsci, latex, latex2e, latex2man, latex3, latexbug, latexindent, latexmk, latex-mr, leaflet, leipzig, libertine, libertinegc, libertinus, libertinust1math, lion-msc, lni, longdivision, lshort-chinese, ltb2bib, lualatex-math, lualibs, luamesh, luamplib, luaotfload, luapackageloader, luatexja, luatexko, lwarp, make4ht, marginnote, markdown, mathalfa, mathpunctspace, mathtools, mcexam, mcf2graph, media9, minidocument, modular, montserrat, morewrites, mpostinl, mptrees, mucproc, musixtex, mwcls, mweights, nameauth, newpx, newtx, newtxtt, nfssext-cfr, nlctdoc, novel, numspell, nwejm, oberdiek, ocgx2, oplotsymbl, optidef, oscola, overlays, pagecolor, pdflatexpicscale, pdfpages, pdfx, perfectcut, pgfplots, phonenumbers, phonrule, pkuthss, platex, platex-tools, polski, preview, program, proofread, prooftrees, pst-3dplot, pst-barcode, pst-eucl, pst-func, pst-ode, pst-pdf, pst-plot, pstricks, pstricks-add, pst-solides3d, pst-spinner, pst-tools, pst-tree, pst-vehicle, ptex2pdf, ptex-base, ptex-fontmaps, pxbase, pxchfon, pxrubrica, pythonhighlight, quran, ran_toks, reledmac, repere, resphilosophica, revquantum, rputover, rubik, rutitlepage, sansmathfonts, scratch, seealso, sesstime, siunitx, skdoc, songs, spectralsequences, stackengine, stage, sttools, studenthandouts, svg, tcolorbox, tex4ebook, tex4ht, texosquery, texproposal, thaienum, thalie, thesis-ekf, thuthesis, tikz-kalender, tikzmark, tikz-optics, tikz-palattice, tikzpeople, tikzsymbols, titlepic, tl17, tqft, tracklang, tudscr, tugboat-plain, turabian-formatting, txuprcal, typoaid, udesoftec, uhhassignment, ukrainian, ulthese, unamthesis, unfonts-core, unfonts-extra, unicode-math, uplatex, upmethodology, uptex-base, urcls, variablelm, varsfromjobname, visualtikz, xassoccnt, xcharter, xcntperchap, xecjk, xepersian, xetexko, xevlna, xgreek, xsavebox, xsim, ycbook.

Simon Josefsson: OpenPGP smartcard under GNOME on Debian 9.0 Stretch

I installed Debian 9.0 Stretch on my Lenovo X201 laptop today. Installation went smooth, as usual. GnuPG/SSH with an OpenPGP smartcard I use a YubiKey NEO does not work out of the box with GNOME though. I wrote about how to fix OpenPGP smartcards under GNOME with Debian 8.0 Jessie earlier, and I thought I d do a similar blog post for Debian 9.0 Stretch . The situation is slightly different than before (e.g., GnuPG works better but SSH doesn t) so there is some progress. May I hope that Debian 10.0 Buster gets this right? Pointers to which package in Debian should have a bug report tracking this issue is welcome (or a pointer to an existing bug report). After first login, I attempt to use gpg --card-status to check if GnuPG can talk to the smartcard.

jas@latte:~$ gpg --card-status
gpg: error getting version from 'scdaemon': No SmartCard daemon
gpg: OpenPGP card not available: No SmartCard daemon
jas@latte:~$

This fails because scdaemon is not installed. Isn t a smartcard common enough so that this should be installed by default on a GNOME Desktop Debian installation? Anyway, install it as follows.

root@latte:~# apt-get install scdaemon

Then try again.

jas@latte:~$ gpg --card-status
gpg: selecting openpgp failed: No such device
gpg: OpenPGP card not available: No such device
jas@latte:~$

I believe scdaemon here attempts to use its internal CCID implementation, and I do not know why it does not work. At this point I often recall that want pcscd installed since I work with smartcards in general.

root@latte:~# apt-get install pcscd

Now gpg --card-status works!

jas@latte:~$ gpg --card-status
Reader ...........: Yubico Yubikey NEO CCID 00 00
Application ID ...: D2760001240102000006017403230000
Version ..........: 2.0
Manufacturer .....: Yubico
Serial number ....: 01740323
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Sex ..............: male
URL of public key : https://josefsson.org/54265e8c.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: rsa2048 rsa2048 rsa2048
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 8358
Signature key ....: 9941 5CE1 905D 0E55 A9F8  8026 860B 7FBB 32F8 119D
      created ....: 2014-06-22 19:19:04
Encryption key....: DC9F 9B7D 8831 692A A852  D95B 9535 162A 78EC D86B
      created ....: 2014-06-22 19:19:20
Authentication key: 2E08 856F 4B22 2148 A40A  3E45 AF66 08D7 36BA 8F9B
      created ....: 2014-06-22 19:19:41
General key info..: sub  rsa2048/860B7FBB32F8119D 2014-06-22 Simon Josefsson 
sec#  rsa3744/0664A76954265E8C  created: 2014-06-22  expires: 2017-09-04
ssb>  rsa2048/860B7FBB32F8119D  created: 2014-06-22  expires: 2017-09-04
                                card-no: 0006 01740323
ssb>  rsa2048/9535162A78ECD86B  created: 2014-06-22  expires: 2017-09-04
                                card-no: 0006 01740323
ssb>  rsa2048/AF6608D736BA8F9B  created: 2014-06-22  expires: 2017-09-04
                                card-no: 0006 01740323
jas@latte:~$

Using the key will not work though.

jas@latte:~$ echo foo gpg -a --sign
gpg: no default secret key: No secret key
gpg: signing failed: No secret key
jas@latte:~$

This is because the public key and the secret key stub are not available.

jas@latte:~$ gpg --list-keys
jas@latte:~$ gpg --list-secret-keys
jas@latte:~$

You need to import the key for this to work. I have some vague memory that gpg --card-status was supposed to do this, but I may be wrong.

jas@latte:~$ gpg --recv-keys 9AA9BDB11BB1B99A21285A330664A76954265E8C
gpg: failed to start the dirmngr '/usr/bin/dirmngr': No such file or directory
gpg: connecting dirmngr at '/run/user/1000/gnupg/S.dirmngr' failed: No such file or directory
gpg: keyserver receive failed: No dirmngr
jas@latte:~$

Surprisingly, dirmngr is also not shipped by default so it has to be installed manually.

root@latte:~# apt-get install dirmngr

Below I proceed to trust the clouds to find my key.

jas@latte:~$ gpg --recv-keys 9AA9BDB11BB1B99A21285A330664A76954265E8C
gpg: key 0664A76954265E8C: public key "Simon Josefsson " imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1
jas@latte:~$

Now the public key and the secret key stub are available locally.

jas@latte:~$ gpg --list-keys
/home/jas/.gnupg/pubring.kbx
----------------------------
pub   rsa3744 2014-06-22 [SC] [expires: 2017-09-04]
      9AA9BDB11BB1B99A21285A330664A76954265E8C
uid           [ unknown] Simon Josefsson 
uid           [ unknown] Simon Josefsson 
sub   rsa2048 2014-06-22 [S] [expires: 2017-09-04]
sub   rsa2048 2014-06-22 [E] [expires: 2017-09-04]
sub   rsa2048 2014-06-22 [A] [expires: 2017-09-04]
jas@latte:~$ gpg --list-secret-keys
/home/jas/.gnupg/pubring.kbx
----------------------------
sec#  rsa3744 2014-06-22 [SC] [expires: 2017-09-04]
      9AA9BDB11BB1B99A21285A330664A76954265E8C
uid           [ unknown] Simon Josefsson 
uid           [ unknown] Simon Josefsson 
ssb>  rsa2048 2014-06-22 [S] [expires: 2017-09-04]
ssb>  rsa2048 2014-06-22 [E] [expires: 2017-09-04]
ssb>  rsa2048 2014-06-22 [A] [expires: 2017-09-04]
jas@latte:~$

I am now able to sign data with the smartcard, yay!

jas@latte:~$ echo foo gpg -a --sign
-----BEGIN PGP MESSAGE-----
owGbwMvMwMHYxl2/2+iH4FzG01xJDJFu3+XT8vO5OhmNWRgYORhkxRRZZjrGPJwQ
yxe68keDGkwxKxNIJQMXpwBMRJGd/a98NMPJQt6jaoyO9yUVlmS7s7qm+Kjwr53G
uq9wQ+z+/kOdk9w4Q39+SMvc+mEV72kuH9WaW9bVqj80jN77hUbfTn5mffu2/aVL
h/IneTfaOQaukHij/P8A0//Phg/maWbONUjjySrl+a3tP8ll6/oeCd8g/aeTlH79
i0naanjW4bjv9wnvGuN+LPHLmhUc2zvZdyK3xttN/roHvsdX3f53yTAxeInvXZmd
x7W0/hVPX33Y4nT877T/ak4L057IBSavaPVcf4yhglVI8XuGgaTP666Wuslbliy4
5W5eLasbd33Xd/W0hTINznuz0kJ4r1bLHZW9fvjLduMPq5rS2co9tvW8nX9rhZ/D
zycu/QA=
=I8rt
-----END PGP MESSAGE-----
jas@latte:~$

Encrypting to myself will not work smoothly though.

jas@latte:~$ echo foo gpg -a --encrypt -r simon@josefsson.org
gpg: 9535162A78ECD86B: There is no assurance this key belongs to the named user
sub  rsa2048/9535162A78ECD86B 2014-06-22 Simon Josefsson 
 Primary key fingerprint: 9AA9 BDB1 1BB1 B99A 2128  5A33 0664 A769 5426 5E8C
      Subkey fingerprint: DC9F 9B7D 8831 692A A852  D95B 9535 162A 78EC D86B
It is NOT certain that the key belongs to the person named
in the user ID.  If you *really* know what you are doing,
you may answer the next question with yes.
Use this key anyway? (y/N) 
gpg: signal Interrupt caught ... exiting
jas@latte:~$

The reason is that the newly imported key has unknown trust settings. I update the trust settings on my key to fix this, and encrypting now works without a prompt.

jas@latte:~$ gpg --edit-key 9AA9BDB11BB1B99A21285A330664A76954265E8C
gpg (GnuPG) 2.1.18; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
pub  rsa3744/0664A76954265E8C
     created: 2014-06-22  expires: 2017-09-04  usage: SC  
     trust: unknown       validity: unknown
ssb  rsa2048/860B7FBB32F8119D
     created: 2014-06-22  expires: 2017-09-04  usage: S   
     card-no: 0006 01740323
ssb  rsa2048/9535162A78ECD86B
     created: 2014-06-22  expires: 2017-09-04  usage: E   
     card-no: 0006 01740323
ssb  rsa2048/AF6608D736BA8F9B
     created: 2014-06-22  expires: 2017-09-04  usage: A   
     card-no: 0006 01740323
[ unknown] (1). Simon Josefsson 
[ unknown] (2)  Simon Josefsson 
gpg> trust
pub  rsa3744/0664A76954265E8C
     created: 2014-06-22  expires: 2017-09-04  usage: SC  
     trust: unknown       validity: unknown
ssb  rsa2048/860B7FBB32F8119D
     created: 2014-06-22  expires: 2017-09-04  usage: S   
     card-no: 0006 01740323
ssb  rsa2048/9535162A78ECD86B
     created: 2014-06-22  expires: 2017-09-04  usage: E   
     card-no: 0006 01740323
ssb  rsa2048/AF6608D736BA8F9B
     created: 2014-06-22  expires: 2017-09-04  usage: A   
     card-no: 0006 01740323
[ unknown] (1). Simon Josefsson 
[ unknown] (2)  Simon Josefsson 
Please decide how far you trust this user to correctly verify other users' keys
(by looking at passports, checking fingerprints from different sources, etc.)
  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu
Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y
pub  rsa3744/0664A76954265E8C
     created: 2014-06-22  expires: 2017-09-04  usage: SC  
     trust: ultimate      validity: unknown
ssb  rsa2048/860B7FBB32F8119D
     created: 2014-06-22  expires: 2017-09-04  usage: S   
     card-no: 0006 01740323
ssb  rsa2048/9535162A78ECD86B
     created: 2014-06-22  expires: 2017-09-04  usage: E   
     card-no: 0006 01740323
ssb  rsa2048/AF6608D736BA8F9B
     created: 2014-06-22  expires: 2017-09-04  usage: A   
     card-no: 0006 01740323
[ unknown] (1). Simon Josefsson 
[ unknown] (2)  Simon Josefsson 
Please note that the shown key validity is not necessarily correct
unless you restart the program.
gpg> quit
jas@latte:~$ echo foo gpg -a --encrypt -r simon@josefsson.org
-----BEGIN PGP MESSAGE-----
hQEMA5U1Fip47NhrAQgArTvAykj/YRhWVuXb6nzeEigtlvKFSmGHmbNkJgF5+r1/
/hWENR72wsb1L0ROaLIjM3iIwNmyBURMiG+xV8ZE03VNbJdORW+S0fO6Ck4FaIj8
iL2/CXyp1obq1xCeYjdPf2nrz/P2Evu69s1K2/0i9y2KOK+0+u9fEGdAge8Gup6y
PWFDFkNj2YiVa383BqJ+kV51tfquw+T4y5MfVWBoHlhm46GgwjIxXiI+uBa655IM
EgwrONcZTbAWSV4/ShhR9ug9AzGIJgpu9x8k2i+yKcBsgAh/+d8v7joUaPRZlGIr
kim217hpA3/VLIFxTTkkm/BO1KWBlblxvVaL3RZDDNI5AVp0SASswqBqT3W5ew+K
nKdQ6UTMhEFe8xddsLjkI9+AzHfiuDCDxnxNgI1haI6obp9eeouGXUKG
=s6kt
-----END PGP MESSAGE-----
jas@latte:~$

So everything is fine, isn t it? Alas, not quite.

jas@latte:~$ ssh-add -L
The agent has no identities.
jas@latte:~$

Tracking this down, I now realize that GNOME s keyring is used for SSH but GnuPG s gpg-agent is used for GnuPG. GnuPG uses the environment variable GPG_AGENT_INFO to connect to an agent, and SSH uses the SSH_AUTH_SOCK environment variable to find its agent. The filenames used below leak the knowledge that gpg-agent is used for GnuPG but GNOME keyring is used for SSH.

jas@latte:~$ echo $GPG_AGENT_INFO 
/run/user/1000/gnupg/S.gpg-agent:0:1
jas@latte:~$ echo $SSH_AUTH_SOCK 
/run/user/1000/keyring/ssh
jas@latte:~$

Here the same recipe as in my previous blog post works. This time GNOME keyring only has to be disabled for SSH. Disabling GNOME keyring is not sufficient, you also need gpg-agent to start with enable-ssh-support. The simplest way to achieve that is to add a line in ~/.gnupg/gpg-agent.conf as follows. When you login, the script /etc/X11/Xsession.d/90gpg-agent will set the environment variables GPG_AGENT_INFO and SSH_AUTH_SOCK. The latter variable is only set if enable-ssh-support is mentioned in the gpg-agent configuration.

jas@latte:~$ mkdir ~/.config/autostart
jas@latte:~$ cp /etc/xdg/autostart/gnome-keyring-ssh.desktop ~/.config/autostart/
jas@latte:~$ echo 'Hidden=true' >> ~/.config/autostart/gnome-keyring-ssh.desktop 
jas@latte:~$ echo enable-ssh-support >> ~/.gnupg/gpg-agent.conf 
jas@latte:~$

Log out from GNOME and log in again. Now you should see ssh-add -L working.

jas@latte:~$ ssh-add -L
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFP+UOTZJ+OXydpmbKmdGOVoJJz8se7lMs139T+TNLryk3EEWF+GqbB4VgzxzrGjwAMSjeQkAMb7Sbn+VpbJf1JDPFBHoYJQmg6CX4kFRaGZT6DHbYjgia59WkdkEYTtB7KPkbFWleo/RZT2u3f8eTedrP7dhSX0azN0lDuu/wBrwedzSV+AiPr10rQaCTp1V8sKbhz5ryOXHQW0Gcps6JraRzMW+ooKFX3lPq0pZa7qL9F6sE4sDFvtOdbRJoZS1b88aZrENGx8KSrcMzARq9UBn1plsEG4/3BRv/BgHHaF+d97by52R0VVyIXpLlkdp1Uk4D9cQptgaH4UAyI1vr cardno:000601740323
jas@latte:~$

Topics for further discussion or research include 1) whether scdaemon, dirmngr and/or pcscd should be pre-installed on Debian desktop systems; 2) whether gpg --card-status should attempt to import the public key and secret key stub automatically; 3) why GNOME keyring is used by default for SSH rather than gpg-agent; 4) whether GNOME keyring should support smartcards, or if it is better to always use gpg-agent for GnuPG/SSH, 5) if something could/should be done to automatically infer the trust setting for a secret key. Enjoy!

Adnan Hodzic: Automagically deploy & run containerized WordPress (PHP7 FPM, Nginx, MariaDB) using Ansible + Docker on AWS

In this blog post, I ve described what started as simple migration of WordPress blog to AWS, ended up as automation project consisting of publishing multiple Ansible roles deploying and running multiple Docker images. If you re not interested in reading about my entire journey, cognition gains and how this process came to be, please skim down to Birth of: containerized-wordpress-project (TL;DR) section. Migrating WordPress blog to AWS (EC2, Lightsail?) Since I ve been sold on Amazon s AWS idea of cloud computing services for couple of years now. I ve wanted, and been trying to migrate this (WordPress) blog to AWS, but somehow it never worked out. Moving it to EC2 instance, with its own ELB volumes, AMI, EIP, Security Group it just seemed as an overkill. When AWS Lightsail was first released, it seemed that was an answer to all my problems. But it wasn t, disregarding its bit restrictive/dumbed down versions of original features. Living in Amsterdam, my main problem with it was that it was only available in a single US region. Regardless, I thought it had everything I needed for WordPress site, and as a new service, it had great potential. Its regional limitations were also good in a sense that they made me realize one important thing. And that s once I migrate my blog to AWS, I want to be able to seamlessly move/migrate it across different EC2 s and different regions once they were available. If done properly, it meant I could even have it moved across different clouds (I m talking to you Google Cloud). P.S: AWS Lightsail is now available in couple of different regions across Europe. Rollout which was almost smoothless. Fundamental problem of every migration is migration

Phase 1: Don t reinvent the wheel? When you have a WordPress site that s not self hosted. You want everything to work, but yet you really don t want to spend any time managing infrastructure it s on. And as soon as I started looking what could fit this criteria, I found that there were pre-configured, running out of box WordPress EC2 images available on AWS Marketplace, great! But when I took a look, although everything was running out of box, I wasn t happy with software stack it was all built on. Namely Ubuntu 14.04 and Apache, and all of the services were started using custom scripts. Yuck. With this setup, when it was time to upgrade (and it s already that time) you wouldn t be thinking about upgrade. You d only be thinking about another migration. Phase 2: What if I built everything myself? Installing and configuring everything manually, and then writing huge HowTo which I would follow when I needed to re-create whole stack was not an option. Same case with was scripting whole process, as overhead of changes that had to be tracked was too way too big. Being a huge Ansible fan, automating this step was a natural next step. I even found an awesome Ansible role which seemed like it s going to do everything I need. Except, I realized I needed to update all software that s deployed with it, and customize it since configuration it was deployed on wasn t as generic. So I forked it and got to work. But soon enough, I was knee deep in making and fiddling with various system changes. Something I was trying to get away in this case, and most importantly something I was trying to avoid when it was time for next update. Phase 3: Marriage made in heaven: Ansible + Docker + AWS Idea to have everything Dockerized was around from very start. However, it never made a lot of sense until I put Ansible into same picture. And it was at this point where my final idea and requirements become crystal clear. Use Ansible to configure and setup host ready for Docker ecosystem. Ecosystem consisting of multiple separate containers for each required service (WordPress + Nginx + MariaDB). Link them all together as a single service using Docker Compose. Idea was backed by thought to spend minimum to no time (and effort) on manual configuration of anything on the server. Level of attachment to this server was so low that I didn t even want to SSH to it. If there was something wrong, I could just nuke the whole thing and deploy code on a new healthy rolled out server with everything working out of box. After it was clear what needed to be done, I got to work.

Birth of: containerized-wordpress-project (TL;DR)

After a lot of work, end result is project which allows you to automagically deploy & run containerized WordPress instance which consists of 3 separate containers running:

WordPress (PHP7 FPM)
Nginx
MariaDB

Once run, containerized-wordpress playbook will guide you through interactive setup of all 3 containers, after which it will run all Ansible roles created for this project. End result is that host you have never even SSH-ed to will be fully configured and running containerized WordPress instace out of box.

Most importantly, this whole process will be completed in <= 5 minutes and doesn t require any Docker or Ansible knowledge! containerized-wordpress demo Console output of running containerized-wordpress Ansible Playbook:

Accessing WordPress instance created from containerized-wordpress Ansible Playbook:

Did I end up migrating to AWS in the end? You bet. Thanks to efforts made in containerized-wordpress-project, I m happy to report my whole WordPress migration to AWS was completed in matter of minutes and that this blog is now running on Docker and on AWS! I hope this same project will help you take a leap in your migration. Happy hacking!

Search Results: "dz"

4 May 2021

11 April 2021

Major monasteries Rumtek Monastery, 20Km from Gangtok Lingdum/Ranka Monastery, 17Km from Gangtok Phodong Monastery, 28Km from Gangtok Ralang Monastery, 10Km from Ravangla Tsuklakhang Monastery, Royal Palace, Gangtok Enchey Monastery, Gangtok Tashiding Monastery, 35Km from Ravangla

East Sikkim

West Sikkim Time needed: 3N/1N Hostels at Pelling : Mochilerro Ostillo

Itinerary

Day 1: Gangtok - Ravangla - Pelling Leave Gangtok early, for Ravangla through the Temi Tea Estate route. Spend some time at the tea garden and then visit Buddha Park at Ravangla Head to Pelling from Ravangla

Day 2: Pelling sightseeing Hire a cab and visit Skywalk, Pemayangtse Monastery, Rabdentse Ruins, Kecheopalri Lake, Kanchenjunga Falls.

Day 3: Pelling - Gangtok/Siliguri Wake up early to catch a glimpse of Kanchenjunga at the Pelling Helipad around sunrise Head back to Gangtok on a shared-cab You could take a bus/taxi back to Siliguri if Pelling is your last stop.

Itinerary

Shared-cab hacks Intercity rides can be exhausting. If you can afford it, pay for an additional seat. Call shotgun on the drives beyond Lachen and Lachung. The views are breathtaking. Return cabs tend to be cheaper (WB cabs travelling from SK and vice-versa)

Buddhist souvenirs : Colourful Prayer Flags (great for tying on bikes or behind car windshields) Miniature Prayer/Mani Wheels Lucky Charms, Pendants and Key Chains Cham Dance masks and robes Singing Bowls Common symbols: Om mani padme hum, Ashtamangala, Zodiac signs

Handicrafts & Handlooms Tibetan Yak Wool shawls, scarfs and carpets Sikkimese Ceramic cups Thangka Paintings

Edibles Darjeeling Tea (usually brewed and not boiled) Wine (Arucha Peach & Rhododendron) Dalle Khursani (Chilli) Paste and Pickle

31 March 2021

7 March 2021

16 November 2020

1 November 2020

30 September 2020

Tools

Beamer (LaTeX) LaTeX class Do not use directly unless you are a LaTeX expert or masochist, see Pandoc below see also powerdot Home page

Darkslide HTML, Javascript presenter notes, table of contents, Markdown, RST, Textile, themes, code samples, auto-reload Home page, demo

Impress.js Javascript Zooms in and out, 3D support Source code, demo Hekyll uses Jekyll as a backend

Impressive simply displays PDFs or images page transitions, overview screen, highlighting Home page

Libreoffice Impress Powerpoint clone Makes my life miserable PDF export, presenter notes, outline view, etc Home page, screenshots

Magicpoint ancestor of everyone else (1997!) text input format, image support, talk timer, slide guides, HTML/Postscript export, draw on slides, X11 output no release since 2008 Home page

mdp and lookatme (commandline) Commandline-only, markdown Home page lookatme is similar

Pandoc Allows converting from basically whatever into slides, including Beamer, DZSlides, reveal.js, slideous, slidy, Powerpoint PDF, HTML, Powerpoint export, presentation notes, full screen background images nice plain text or markdown input format Home page, documentation

PDF Presenter PDF presentation tool, shows presentation notes basically "Keynote for Linux" Home page, pdf-presenter-console in Debian

Pinpoint Native GNOME app Full screen slides, PDF export, live change, presenter notes, pango markup, video, image backgrounds Home page Abandoned since at least 2019

Reveal.js HTML, Javascript PDF export, Markdown, LaTeX support, syntax-highlighting, nested slides, speaker notes Source code, demo

S5 HTML, CSS incremental, bookmarks, keyboard controls can be transformed from ReStructuredText (RST) with rst2s5 with python-docutils Home page, demo

sent X11 only plain text, black on white, image support, and that's it from the suckless.org elitists Home page

Sozi Entire presentation is one poster, zooming and jumping around SVG + Javascript Home page, demo

2 June 2020

The importance of early pneumonia detection

The project from a technical standpoint

General system description

The hardware

The software

Certifications

Funding

17 May 2020

27 April 2020

15 March 2020

12 March 2020

6 November 2017

4. Things Happen It's tricky to pick a track from either "Love's Secret Domain" or "Horse Rotorvator"; there are other choices which I think are better known and loved than this one but it's one that haunted me after I first heard it for one reason or another, so here it is.

28 September 2017

10 September 2017

31 August 2017

29 August 2017

20 June 2017

18 June 2017

21 May 2017

Major monasteries

Rumtek Monastery, 20Km from Gangtok

Lingdum/Ranka Monastery, 17Km from Gangtok

Phodong Monastery, 28Km from Gangtok

Ralang Monastery, 10Km from Ravangla

Tsuklakhang Monastery, Royal Palace, Gangtok

Enchey Monastery, Gangtok

Tashiding Monastery, 35Km from Ravangla

West Sikkim

Time needed: 3N/1N

Hostels at Pelling : Mochilerro Ostillo

Day 1: Gangtok - Ravangla - Pelling

Leave Gangtok early, for Ravangla through the Temi Tea Estate route.

Spend some time at the tea garden and then visit Buddha Park at Ravangla

Head to Pelling from Ravangla

Day 2: Pelling sightseeing

Hire a cab and visit Skywalk, Pemayangtse Monastery, Rabdentse Ruins, Kecheopalri Lake, Kanchenjunga Falls.

Day 3: Pelling - Gangtok/Siliguri

Wake up early to catch a glimpse of Kanchenjunga at the Pelling Helipad around sunrise

Head back to Gangtok on a shared-cab

You could take a bus/taxi back to Siliguri if Pelling is your last stop.

Shared-cab hacks

Intercity rides can be exhausting. If you can afford it, pay for an additional seat.

Call shotgun on the drives beyond Lachen and Lachung. The views are breathtaking.

Return cabs tend to be cheaper (WB cabs travelling from SK and vice-versa)

Buddhist souvenirs :

Colourful Prayer Flags (great for tying on bikes or behind car windshields)

Miniature Prayer/Mani Wheels

Lucky Charms, Pendants and Key Chains

Cham Dance masks and robes

Singing Bowls

Common symbols: Om mani padme hum, Ashtamangala, Zodiac signs

Handicrafts & Handlooms

Tibetan Yak Wool shawls, scarfs and carpets

Sikkimese Ceramic cups

Thangka Paintings

Edibles

Darjeeling Tea (usually brewed and not boiled)

Wine (Arucha Peach & Rhododendron)

Dalle Khursani (Chilli) Paste and Pickle

Beamer (LaTeX)

LaTeX class

Do not use directly unless you are a LaTeX expert or masochist, see Pandoc below

see also powerdot

Home page

Darkslide

HTML, Javascript

presenter notes, table of contents, Markdown, RST, Textile, themes, code samples, auto-reload

Home page, demo

Impress.js

Javascript

Zooms in and out, 3D support

Source code, demo

Hekyll uses Jekyll as a backend

Impressive

simply displays PDFs or images

page transitions, overview screen, highlighting

Home page

Libreoffice Impress

Powerpoint clone

Makes my life miserable

PDF export, presenter notes, outline view, etc

Home page, screenshots

Magicpoint

ancestor of everyone else (1997!)

text input format, image support, talk timer, slide guides, HTML/Postscript export, draw on slides, X11 output

no release since 2008

Home page

mdp and lookatme (commandline)

Commandline-only, markdown

Home page

lookatme is similar

Pandoc

Allows converting from basically whatever into slides, including Beamer, DZSlides, reveal.js, slideous, slidy, Powerpoint

PDF, HTML, Powerpoint export, presentation notes, full screen background images

nice plain text or markdown input format

Home page, documentation

PDF Presenter

PDF presentation tool, shows presentation notes

basically "Keynote for Linux"

Home page, pdf-presenter-console in Debian

Pinpoint

Native GNOME app

Full screen slides, PDF export, live change, presenter notes, pango markup, video, image backgrounds

Home page

Abandoned since at least 2019

Reveal.js

HTML, Javascript

PDF export, Markdown, LaTeX support, syntax-highlighting, nested slides, speaker notes

Source code, demo

S5

HTML, CSS

incremental, bookmarks, keyboard controls

can be transformed from ReStructuredText (RST) with `rst2s5` with python-docutils

Home page, demo

sent

X11 only

plain text, black on white, image support, and that's it

from the suckless.org elitists

Home page

Sozi

Entire presentation is one poster, zooming and jumping around

SVG + Javascript

Home page, demo